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ABSTRACT 


We have conducted a study on the development of detailed climatological 
probabilities of violating cloud related Lightning Launch Commit Criteria (LLCC) 
used by Cape Canaveral Air Force Station and Kennedy Space Center (CCAFS 
and KSC). This study was conducted to provide the 45th Weather Squadron with 
improved capabilities for operational forecasting for launches from CCAFS and 
KSC. Our focus was on developing methods to produce climatological 
probabilities of violating one of the LLCC, the thick cloud layer rule. We 
developed a hybrid process of blending data from the Climate Forecast System 
Reanalysis (CFSR), meteorological aerodrome reports (METARs), radiosonde 
observations (RAOBs), and expert meteorologist data sets to create a merged 
data set for determining the probability of violating the thick cloud layer rule. 


Using our blended hybrid process, we computed cloud thicknesses, and 
probabilities of violating the thick cloud LLCC for each day of the year at 00Z and 
12Z. Additionally, we conducted a sensitivity analysis to identify the potential for 
modifying the thick cloud LLCC. A primary result from our study is a sub-daily 
data set of the climatological probabilities of violating the thick cloud layer rule. 
We conducted eight validation case studies that demonstrated our calculated 
violations match well with observed violations. The development of a merged 
data set that provides more useful information than any one of the individual data 
sets is a technique that is likely to be useful in solving many other climatological 


problems. 
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I. INTRODUCTION 


A. BACKGROUND 


One of man’s greatest scientific accomplishments is successfully 
launching a rocket from Earth into space. U.S military operations around the 
world are dependent upon these rockets to deliver critical payloads, such as 
navigation, communication, and weather satellites. Along with this incredible 
feat, there are obvious dangers and hazards. This places safety at the absolute 
forefront of space flight. Many of the potentially unsafe conditions are based 
upon varying meteorological parameters. These can range from something as 
simple as a temperature constraint, a wind direction and speed limitation, to 
potential for lightning occurrence, or a specific cloud formation surrounding the 
launch pads. These restrictions are in place to reduce the risk of any adverse 
effects to the mission, in this case the launch of a space vehicle, manned or 
unmanned, to space. The 45th Weather Squadron (45 WS) at Patrick Air Force 
Base, Florida is assigned the task of mitigating the risk of any negative weather 
impacts to launches from Cape Canaveral Air Force Station (CCAFS), and 
Kennedy Space Center (KSC). This team of weather professionals works around 
the clock to provide every launch agency and customer that uses CCAFS and 
KSC for launch operations with timely and critical weather information. The 45 
WS is actively involved in all phases of launch operations: generation, execution, 
and recovery. All phases are equally important to a successful mission; 
however, weather impacts during the execution phase present the greatest risk. 
Any negative impact could potentially lead to a catastrophic launch failure. 
Mission failure could mean the loss of millions of invested dollars, thousands of 
preparation hours, a priceless payload, or worse, the lives of those supporting 
the launch operation. 


A study compiled by the 45 WS for CCAFS/KSC from 1988 through 2006 
revealed 30 percent of all launch operations from CCAFS/KSC are either delayed 


or canceled due to weather impacts (Roeder and McNamara 2006). Weather is 
1 


one of the key factors launch decision makers inquire about in weeks and days 
leading up to a launch. Launch operations are time sensitive, and many 
decisions made are based in large part on weather forecasts provided by the 45 
WS. The 45 WS assigns a specific weather professional to each launch platform 
(e.g., Delta, Atlas, Falcon, etc.). These squadron members are identified as 
launch weather officers (_WOs). Each LWO is in direct communication with the 
launch customer or agency on a daily basis. The team at the 45 WS begins their 
official forecast process as far out as seven days prior to launch, but they also 
receive requests weeks to months in advance to provide climatological planning 
information. This information may steer mission planners to alter their mission 


schedule. 


The 45 WS members have numerous criteria or thresholds they must 
consider when making their forecasts. They range from user-defined constraints, 
such as wind or temperature limitations during fueling, or range safety defined 
constraints, which include a complex set of Lightning Launch Commit Criteria 
(LLCC). The LLCC are a set of rules that outline meteorological conditions that 
could potentially lead to a lightning related hazardous situation during launch. 
Violations of these LLCC are the largest source of negative weather impacts to 
space launches (Roeder and McNamara 2006, FAA 2003). Accurately predicting 
these conditions can potentially save up to millions of dollars, and more 
importantly save lives by mitigating a potential catastrophic loss of mission or 
personnel (Roeder et al. 1999). 


B. LIGHTNING LAUNCH COMMIT CRITERIA 
1. History 


Early in the spaceflight era, rules were in place prohibiting any vehicle 
from launching through a thunderstorm (Merceret et al. 2010). This was the only 
official guidance to weather personnel providing support to launch agencies 
during missions, but this changed after the 1969 launch of Apollo XII. During the 
Apollo XII countdown, there was no observed natural lightning reported near 
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CCAFS/KSC, but an approaching cold front with numerous cumuliform clouds 
was approaching the area. These clouds were extremely large, with tops 
exceeding the freezing level height. Understanding the relationship between 
clouds and freezing level heights will prove to be an important concept in later 
discoveries, as it will lead to re-defining the set of evaluated LLCC. Note in 
Figure 1 that while there was a large cumulus cloud observed immediately over 
the launch pad, it was not a thunderstorm. Therefore, it was not considered a 


threat to the launch. 
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CLOUD PROCESSES ACTING 


Figure 1. Depiction of meteorological conditions near the site of the Apollo XII 
incident (From: Merceret et al. 2010) 


As the Saturn V rocket with the manned space capsule ascended, it 
experienced two lighting strikes in a span of 52 seconds (Merceret et al. 2010). 
The jolt severed communication between the vehicle’s command module 


navigation system, and the mission’s control center. Apollo XII avoided potential 
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catastrophe by regaining communication and successfully completed its objective 
of landing on the moon. However, this event would permanently alter weather 
support to spaceflight operations by generating more restrictive rules for LWOs to 
monitor during launches (Merceret et al. 2010). 


Immediately following the Apollo XII incident, scientists in the meteorology 
and spaceflight fields recognized the Apollo XII events were significant, and 
began taking precautions to protect space vehicles, and explore better 
capabilities for predicting when a similar incident could occur (NASA Facts 1998). 
The outcome was a new set of LLCC for the weather teams to monitor during 


launches. 


By 1987, scientists had created a more restrictive set of LLCC. The 


newest rules directed that a space vehicle could not launch: 
1) Through a thunderstorm/cumulonimbus cloud 


2) Within 5 miles of a thunderstorm/cumulonimbus cloud or 3 miles of 


the associated anvil top 


3) Through a cold front or squall line associated clouds with tops 
10,000 feet or higher 


4) Through middle cloud layers, 6000 feet or greater in depth, where 
the freezing level is in the clouds 


5) Through cumulus clouds where the freezing level is in the clouds 


Despite these newly modified LLCC, Atlas/Centaur 67 (AC 67) 
experienced a similar situation as Apollo XII, but with greatly different results. As 
the AC 67 vehicle began its ascent, it also experienced a lightning strike in 
conditions where no natural lightning was observed. Only a few seconds after 
this strike, the vehicle showed signs of steering off the nominal course, and 
began breaking apart. The range safety officer sent the self-destruct signal, 
destroying the vehicle and its Fleet SatCom satellite payload, but protecting lives. 
The destruction of the AC 67 vehicle generated a large inquiry into the cause of 
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the event. One of the significant outcomes was the creation of the Lightning 
Review Committee (LRC), which later became the Lightning Advisory Panel 
(LAP). This is a group of research scientists in the fields of space flight, 
meteorology, and atmospheric electricity. Their objective was to come to a 
consensus, based on their varying expertise, and determine what atmospheric 
conditions led to the destruction of AC 67 (Merceret et al., 2010). 





Figure 2. Triggered lightning strike to AC-67 in 1987. Only seconds after liftoff, the 
lightning followed the exhaust plume to the ground. (From: Roeder and 
McNamara 2006) 


These two historical events highlight the significance of understanding 
induced lightning generated from a launched vehicle, versus launching in 
conditions where natural lightning is observed. This induced, or triggered, 
lightning is the basis for the majority of the existing LLCC, and represents the 


initial focus for our research. 


2. Triggered Lightning 


Discerning between natural and triggered lightning is critically important; 
eleven of the twelve current LLCC are in place to mitigate the risk of triggered 
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lightning. Natural lightning is lightning produced by thunderstorms, while 
triggered lightning results when a rocket is launched into a pre-existing and 
sufficiently strong electric field (Roeder and McNamara 2006). Additionally, the 
exhaust plume that extends from the rocket is electrically conductive, and further 
adds to the potential for a triggered lightning discharge. 


UNDISTURBED FIELD DISTURBED STATE 
inal =e 


a) Larger than normal electric field from an b) Rocket launch, especially from the long 
atmospheric source, not specified. Lines are conductive exhaust plume, increases the pre- 
isopleths of equal electric potential. existing electric field/potential gradient. 


c) Breakdown of the electric field/potential 
gradient of air is met or exceeded, resulting in 
triggered lightning 





Figure 3. Schematic of the rocket triggered lightning process ( From: 45 WS 2009) 


In an undisturbed state, the electric field surrounding the launch pad 
typically exhibits an even spacing of electric potential values (Figure 3a). Once 
the rocket launches, compression of the electric field can ensue (Figure 3b). If 
the compression of the electric field continues and reaches or exceeds the 


existing electric field gradient, a triggered lightning strike can occur (Figure 3c) 
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(Roeder and McNamara 2006). Rocket triggered lightning can occur in electric 
fields that are tens to hundreds of times smaller than what is required to produce 
natural lightning (Merceret et al. 2010). The existing LLCC are in place primarily 
to protect against electric charge generated because of launching into mixed 
solid-liquid phase of water (Roeder and McNamara 2006). As seen in previous 
historical events, launching into non-thunderstorm clouds can still produce 
lightning if certain meteorological conditions exist. Many of those conditions will 
deal with clouds that meet specific restrictions, such as proximity, depth, type, 
and temperatures within the cloud. Accurately measuring these parameters 
enables the weather team to assess the potential for triggered lightning. Each 
restriction that is measured provides important information for defining the 
electric field, or inferring the existence of a mixed phase of water. From 1969-— 
2010, the LLCC have undergone numerous modifications based on new scientific 
information. A summary of the changes is located in the NASA document, “A 
History of the Lightning Launch Commit Criteria and the Lightning Advisory Panel 
for America’s Space Program” (Merceret et al. 2010). Since Apollo XIl, the 
LLCCs have evolved from a singularly evaluated LLCC (cannot launch in a 
thunderstorm), to the present complex set of ten rules. 


a: Operational Set of LLCC 


When 45 WS LWOs prepare their forecast for upcoming missions, they 
need to assess the likelihood an LLCC may be violated during the launch 
countdown. This set of LLCCs is what LWOs actually evaluate and report as 
“No-Go” (violated), or “Go” (non-violated) to critical mission personnel during 
countdowns. The latest revision to the LLCC was accepted and put into 
operational use in 2009. The full set of LLCC evaluated during all launches from 
CCAFS/KSC is listed below. 


1) Surface Electric Field Mill 
2) Natural Lightning 


3) Cumulus Clouds 


4) Attached Anvil Clouds 
5) Detached Anvil Clouds 
6) Debris Clouds 

7) Disturbed Weather 

8) Thick Cloud Layer 

9) Smoke Plumes 

10) Triboelectrification 


Each criterion is divided into sub categories based on various parameters, 
such as temperatures, standoff distances, and time requirements. Some of 
these rules also have caveats, which allow for relief from violation of the main 
rule if certain meteorological conditions are met. A complete description and 
details of all the currently applied LLCC are contained in the Air Force Space 
Command Manual 91-710 (Air Force Space Command 2004). 


C. RESEARCH MOTIVATION/SCOPE 
1. Prior Work 


Over the years, numerous studies were conducted documenting the 
natural lighting portions of the LLCC. However, there are relatively few studies 
assessing the probability of violating, and building climatologies of, the LLCC, 
especially those dealing with cloud-based criteria. Muller (2010) adequately 
summarizes much of the previous work done in regards to the natural lightning 
LLCC. 


Goetz (2000) explored the possibility of building hourly and seasonal 
climatologies of the natural lightning and cumulus cloud LLCC. His work focused 
on data from 1989-1998, and within a 12 nm radius of the CCAFS/KSC region. 
The output of his results was an unsmoothed hourly probability of violation for 
each day of every month and year during the timespan of his study. However, 


cloud type climatology is relatively difficult to assess given the nature of surface 
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weather observations, and lack of cloud type input (Taniguchi 2005). This 
presented some limitations in accurately depicting probabilities of violating the 
cumulus cloud LLCC. Additionally, his results indicated very large day-to-day 
variations of the probabilities, possibly due to the relatively short period of record 
(1989-1998). While this was an early attempt at building LLCC climatology, it did 
not deliver an operational product the 45 WS could use when preparing their 


customers for a launch. 


Muller (2010) assessed the probabilities of violating the natural lightning 
LLCC and developed a daily climatology of the probability of violating the natural 
lightning LLCC. He also conducted a climate analysis of natural lighting around 
the CCAFS/KSC region and explored interannual and seasonal variations of 
conditions related to natural lightning using atmospheric reanalysis data. Of 
note, Muller highlighted the need for a more advanced approach to building 
climatologies for more of the LLCC. 


2. Research Focus 


The 45 WS has stressed the importance and significance of assessing the 
triggered lightning LLCC to develop useful, and usable, operational tools the 
LWOs can refer to when making their forecasts (Roeder, personal 
communication). There are millions of dollars invested, thousands of resources 
used, and countless hours spent in preparation of a CCAFS/KSC launch. Safety 
is paramount, and mitigating the risk of any catastrophic failure, like AC 67, is 
paramount when 45 WS personnel are making their predictions of violating the 
LLCC. 


Addressing all of the LLCC would be too difficult for the scope of this 
research, so we decided to assess several factors to determine where to start. 
Variables considered in determining our focus were: (a) frequency of violation; 
(b) false alarm rate; and (c) 45 WS LLCC priority ranking (a ranked set of LLCC 
based on input from 45 WS members and key operators). All of these factors, 
coupled with input from the 45 WS directly, led us to focus our research on the 
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thick cloud layer LLCC. There is no existing complete data base from which the 
definitive probability of violating one of the LLCC rules can be directly 
determined. Therefore, in this study, we explored the blending of multiple data 
sets to develop a merged data set from which the probability of violating the thick 
cloud layer rule could be directly calculated. We anticipate that future research 


efforts will be able to apply similar techniques to other LLCC. 
Our research was primarily focused on the following topics: 


1) Determine the best approach to analyzing and synthesizing multiple 
data sets to infer cloud thicknesses and the associated applications 


of the thick cloud layer rule 
2) Develop a climatological data base of the thick cloud layer LLCC 


3) Build a useful and meaningful metric, such as a probability of 
violation (POV) for the thick cloud layer rule. 


4) Conduct a sensitivity analysis of our resulting POVs for use in 
determining future applications or modifications of the thick cloud 


layer rule. 


3. Thesis Organization 


The topics outlined above are addressed in this thesis through a 
systematic approach. Chapter Il describes our study region, study period, a 
typical approach the 45 WS employs in evaluating the LLCC during a launch, 
details of the thick cloud rule, datasets used, and the methodology we applied to 
our data sets. Chapter Ill outlines the results from our analyses of the data sets, 
our blending of the data sets, and our calculations of the climatological 


probabilities of violating the thick cloud layer LLCC. 


Additionally, Chapter III contains the results of our analysis of the sensitivity of 
the thick cloud layer POVs to variations in the thickness threshold, and a small 
sample case study of known violations of the thick cloud layer rule. Chapter IV 
provides a summary of our results, conclusions we have made from them, and 


suggestions for future research. 
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ll. DATA AND METHODS 


A. STUDY REGION AND PERIOD 


The CCAFS/KSC region covers nearly 500 square miles along the “Space 
Coast” of Florida (Figure 4). The launch pads are located along the immediate 
coast, and are generally separated by approximately 1 nm. Each LLCC is 
evaluated at the launch complex being used for a particular mission. 
Additionally, many of the LLCC specify a standoff distance from the launch site 
within which the rule must also be evaluated. The thick cloud rule specifies that 
the rule must be evaluated within 5 nm of the launch pad being used for a 
particular mission (Figure 5). 


Average Launch Site: 
Latitude: 28.5592°N; 
Longitude: 80.5756°W 





Figure 4. Map indicating five nautical mile ring around the average launch site (light 
yellow star). The average launch site is the average of the most active 
launch pads as of September 2011 (Pad 37, Pad 40, and Pad 41). Image 
created from GPS Visualizer [accessed online at 
http://www.gpsvisualizer.com/calculators, Sept 2011]. 
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Keeping in mind the spatial limitations of observed and modeled data, we 
wanted to ensure we captured a large enough sample to allow us to depict the 
climatology over any of the launch sites at CCAFS or KSC. We determined the 
best approach was to focus our research on an average launch site location 
based on the locations of the most active launch pads within CCAFS/KSC. This 
is similar to the approach Muller (2010) used in his natural lightning LLCC 
research. We chose the average launch site location based on the most active 
pads as of September 2011. The location of this average site is 28.5592° north 
latitude and 80.5756° west longitude. With the launch pads being approximately 
1 nm apart, we assumed the meteorological differences between the average 
launch site and the actual launch sites are minimal, and therefore these 


differences could be neglected in our study. 


To develop a robust climatology, we wanted to evaluate meteorological 
conditions at a high temporal and spatial resolution, and over a period of 20 
years or longer, if possible. _ Launches have occurred from the CCAFS/KSC 
region since 1957, and, as described in Chapter |, Apollo XII and AC 67 both 
resulted in major revisions to the LLCCs. With the addition of the LAP, the first 
major revision of the LLCC was put into operational use after AC 67 (Merceret et 
al. 2010). We also needed to consider the availability of the in situ and 
reanalysis meteorological data needed for our study. Balancing all these factors, 
we determined a study period of January 1988—December 2010 would provide 
the optimal combination of data sets. 


B. LLCC 
1. Launch Day Evaluations 


Before identifying what sources of data are available to provide us with 
necessary meteorological information, it is important to understand the process 
the weather team follows during a typical launch countdown. The length of a 
typical launch countdown can vary, but it is typically six hours in duration from 
clock start to liftoff. It is during this time the LWOs evaluate the complete set of 
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current LLCCs. This evaluation process includes the use of weather radar, real 
time satellite data, surface weather observations, weather balloon sounding data, 
several sources of lightning detection data, weather aircraft reports, and input 
from other weather personnel and weather equipment. The typical weather team 
composition on a day of launch is approximately five members, but mission 
requirements may dictate more or less. Each member has an equally important 
role: to monitor and report LLCC violations to the lead LWO, who passes that 
information to launch decision makers. Modern weather instrumentation is 
extremely accurate; however, certain temporal and spatial limitations set in place 
during a launch leads to the need for some interpretation of data by LWOs. The 
published LLCC evaluation documentation states the launch operator must have 
clear and convincing evidence the evaluated LLCC is not violated (Willett et al. 
2010). 


2. Thick Cloud Layer Rule 


As discussed in Chapter |, we determined the focus for our research was 
the thick cloud layer LLCC. The complete description of the thick cloud layer rule 
is summarized below. 

1) A launch operator must not initiate flight if the flight path will carry the 

launch vehicle through: 

a) Acloud layer that is greater than or equal to 4,500 feet thick and 
any part of the cloud layer along the flight path is located at an 
altitude where the temperature is between 0° Celsius and 
-20° Celsius, inclusive ; or 

b) Connected to a thick cloud layer that, if within 5 nm from the 
flight path, is greater than or equal to 4,500 feet thick and has 
any part located at an altitude where the temperature is 
between 0 °Celsius and -20 Celsius, inclusive. 
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This rule does not apply if the thick cloud layer is a cirriform cloud layer that has 
never been associated with convective cloud, is located entirely at altitudes 
where the temperature is less than or equal to -15° Celsius, and shows no 


evidence of containing liquid water. 


This rule is in place because numerous studies have indicated that 
launching in these conditions could result in a rocket-triggered lightning strike 
(Merceret et al. 2010). The sheer complexity of this, or any, LLCC makes it 
difficult for LWOs to make split second decisions during a launch. To assist 
LWOs during training and launch countdowns, the 45 WS developed a set of 
visual aids for each LLCC. Figure 5 depicts a visual representation of part 1.a of 
the thick cloud rule listed above. It demonstrates that if the cloud layer meets the 
requirements outlined, the LWO must report it as “red” (or violated). The entire 
operation becomes “no-go” until the weather team becomes clearly convinced 
conditions that made this rule “red” no longer exist. 





Figure 5. Depiction of an example of a violation of the thick cloud rule (From: 45 WS 
VA 15-3b 2009) 
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Figure 6. Depiction of an example of a violation of the thick cloud rule (From: 45 WS 


VA 15-3b 2009) 


Similarly, Figure 6 refers to part 1.b of the thick cloud rule. The main 
difference is in this case the launch vehicle does not actually penetrate a 4,500 
foot thick cloud layer that meets the temperature requirements, but into a layer 
that is attached to that thick cloud layer within 5 nautical miles. 


C. METEOROLOGICAL VARIABLE SELECTION 


Based on the thick cloud rule definition, we needed to identify key 
variables for which we could obtain data for a sufficiently long period (e.g., 20 
years or longer) and sufficiently high temporal and spatial resolution. There are 
no adequate data sets that directly describe cloud thickness and the other 
variables associated with the thick cloud layer rule (e.g., temperature, cloud 
type). So we needed to obtain data from which we could confidently determine 
cloud thickness. 
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A major initial objective in this part of our research was to evaluate the potential 
data sets for each needed variable and determine its suitability for use in 
developing the merged data set from which we could directly calculate the POVs. 
The main variables needed are summarized in the list below and in Figure 7 (all 
variables are for CCAFS/KSC). 


1) Cloud coverage (whether clouds are located over region or not) 
2) Height of the cloud top 

3) Height of the cloud base 

4) Height of the 0° Celsius level (freezing level) 

5) Height of the -15° Celsius level 


6) Height of the -20° Celsius level 


Height of Cloud Top 


Height of Cloud Base 






Figure 7. Visual representation of the variables required to calculate thick cloud 
layer LLCC violations. 


Once we determined our set of necessary variables, we began identifying 
general aspects and characteristics of each individual variable. One area we 
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needed to evaluate was the average heights of identified temperature levels. 
Information on the average heights of these temperature levels would enable us 
to fine-tune our interrogation of the upper and lower boundaries of cloud layers. 
As the thick cloud LLCC states, the rule is only considered violated if cloud 
thickness is greater than or equal to 4,500 ft, and the temperature stipulations 
are met as well. To initially evaluate these temperature levels, we applied a tool 
used by the 45 WS, and developed by the Applied Meteorology Unit (AMU). The 
AMU is a team of research scientists in the field of meteorology under contract to 
NASA, who work on a daily basis with the 45 WS. They assist the 45 WS with 
creating launch operations related weather tools, one of which is the Range 
Reference Atmosphere (RRA) utility. The RRA ingests twelve years (January 
1990-January 2002) of CCAFS radiosonde observations (RAOBs), and 
calculates monthly and annual climatological averages of specified temperature 
and height levels. Initially, RAOB data is processed at 0.25 km vertical 
resolution, and the AMU-created algorithm interpolates this data to the desired 
temperature or height level. Prior to determining our final set of variable 
conditions, we used the RRA to determine climatological averages of our needed 
temperature levels (0°, -15°, and -20°C). The results are shown in Figures 8 and 
9. 


Temperature Level Climatology 
(Height in feet) 


1990 - 2002 


0° Cee -15° Cee -20° C 























Figure 8. Monthly averages (in feet) of the heights of various temperature levels. 
Averages were calculated using the AMU RRA utility for RAOBs during 
Jan 1990—Dec 2002. The green shaded region represents the vertical 
region in which thick clouds need to exist to lead to a thick cloud rule 
violation. 
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Figure 9. Monthly averages (in pressure) of the heights of various temperature 
levels. Averages were calculated using the AMU RRA utility for RAOBs 
during Jan 1990—Dec 2002. The green shaded region represents the 
vertical region in which thick clouds need to exist to lead to a thick cloud 
rule violation. 
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The shaded green regions of Figures 8 and 9 highlight the vertical region 
within which clouds would need to exist in order to be considered potential 
violators of the thick cloud LLCC. If any part of a cloud is 4,500 ft thick or greater 
and located within these regions, it could potentially cause a launch operation to 
become no-go. We also noted the expected seasonal cycle in the heights: lower 
in colder winter months and higher in warmer summer months. The tool was 
used to calculate the average vertical extents measured in feet and hPa. We 
used this information to evaluate data sets that could potentially provide this 


information on a long-term daily basis. 


The ideal data set for our study would have been one with accurate data 
for all required variables available in a regularly gridded spatial and temporal 
array, at high spatial and temporal resolution, and spanning at least 20 years. 
We realized that such a data set probably did not exist, and we would probably 
need to develop a method for combining multiple data sets together. Therefore, 
we needed to develop processes to identify the strengths and weaknesses of 
various data sets, and determine the best method for combining them into one 
merged data set. 


The four main types of data sets that we used in our study are listed below 
and described in the following sections: 


1. Climate Forecast System Reanalysis (CFSR) data 
2. Meteorological aerodrome report (METAR) data 
3. Radiosonde observations (RAOBs) 


4. Expert meteorologist input data 


D. DATA SETS AND SOURCES 
1. Climate Forecast System Reanalysis (CFSR) 


We considered the many analysis and reanalysis data sets available, and 
determined that CFSR would offer the best choices for the purposes of our study. 


The Climate Forecast System Reanalysis (CFSR, Saha et al. 2010) is a 
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reanalysis data set produced by the National Centers for Environmental 
Prediction (NCEP), and maintained and distributed by the National Center for 
Atmospheric Research (NCAR). It is a high-resolution global data set covering 
the period January 1979-—March 2011 (as of March 2012), and provides data for 
hundreds of atmospheric and oceanic variables at hourly and approximately 0.3 
to 0.5 degree horizontal resolution. CFSR uses a coupled atmosphere-ocean- 
land surface data assimilation and model-based analysis system. The system 
assimilates in situ observations, satellite radiances, and other observational data. 
which are then analyzed using a global dynamical coupled atmosphere-ocean- 
land-ice model. The global atmosphere horizontal resolution is up to ~388km 
(T382), and up to 64 vertical levels from the surface to .26 hPa. It is initialized 
four times per day (00Z, 06Z, 12Z, and 18Z), and products are available in up to 
1 hour intervals. Details about the CFSR data sets we chose for our research 
are provided in later sections. Further details of the CFSR data set and 
processes are provided by Saha etal. (2010). 


a. Variables 


One of our main tasks early in the process of data set selection was 
to sort through the hundreds of variables described by CFSR, including the 
horizontal and vertical locations for which those variables are available. This 
enabled us to determine which variables at which locations we needed to best 
describe the cloud and temperature level conditions over CCAFS and KSC. 
CFSR provided two of our six needed variables directly: cloud coverage and 
height of the 0° C level. It also provided pressure at cloud tops and bottoms, 
which we could use to infer a geometric height. However, the heights of the -15° 
and -20° C levels were not available directly from CFSR. Table 1 summarizes 
the CFSR variables, or parameters, we selected for evaluation and used in our 


study. 
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Parameter 

Total Cloud Cover (%) 

650 hPa 

600 hPa 
500 hPa High Cloud Bottom 500 hPa 


350 hPa a ae 
LevelofO°lsotherm | == ~~~ ‘| LowCloudTop [| = —— 


Middle Cloud Top 
High Cloud Top 
Table 1. |The CFSR variables we used in our thick cloud LLCC study. The levels 


listed in each of the four parameter columns indicates the vertical levels 
for which we obtained and used the CFSR data for that parameter. 
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We used the AMU RRA results (Figures 8-9) to determine the 
levels for which to obtain and evaluate CFSR data. These results gave us the 
climatological means of the heights of the different temperature levels of concern 
(0°, -15°, and -20°C). We also took into account the variations from these means 
to ensure we captured the extreme limits of the heights. The lowest 
climatological height of the 0° C level was 12,311 ft, which equated to a pressure 
surface of 648 hPa. The highest climatological height of the -20° C level was 
26,287 ft, which equated to a pressure surface of 380 hPa. To ensure we 
completely covered the extreme values, we applied two standard deviations and 
rounded off to the nearest 50 hPa increment. This provided us with an overall 
atmospheric boundary of approximately 700-350 hPa for which to obtain and 
evaluate CFSR data. We assumed that clouds occurring within this vertical 
region and which violate the 4,500 ft thickness limit would be likely to also violate 
the thick cloud LLCC because they would also occur within the climatological 


means of the defined temperature thresholds (0° and -20° C). 


b. Spatial and Temporal Resolution 


Although CFSR data is available for the entire globe, we were only 
interested in data at and near CCAFS/KSC. Specifically, we desired data as 
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close as possible to our average launch site (28.5592° north latitude, 80.5756° 
west longitude). Many of the analysis and reanalysis data sets available are for 
large spatial domains, and retrieving information on a smaller scale is sometimes 
difficult. One of the distinct advantages to using CFSR is the ability to select a 
spatial subset of the chosen variables. This allowed us to tune our reanalysis 
data set selection to focus on the Florida Peninsula, and ultimately CCAFS/KSC. 
Additionally, there were multiple horizontal resolution choices ranging from 0.3° 
to 2.5°. Our region of interest is located along a coast, so we wanted to ensure 
we chose a resolution fine enough to capture the coastal conditions as much as 
possible. We evaluated all choices and determined using the 0.5° horizontal 
resolution offered the greatest benefits for the purposes of this study. The 0.5° 
degree mesh places a grid point at 28.5°N latitude, and 80.5°W longitude, very 
close to the average launch site. In addition, data at this resolution is available 
on a standard latitude and longitude grid system, whereas the finer 0.3° 
resolution is placed on a Gaussian grid system. Using the Gaussian grid system 
was certainly feasible; however, in order to maintain consistency with our other 
data sets, we chose to use data on the evenly spaced latitude and longitude 0.5° 


grid system. 


In addition to knowing the closest grid point to our average launch 
site, it was important to have the ability to expand outward to identify differences 
at other grid points. With CCAFS and KSC being along the coast, there is 
obvious potential for land water differences to influence the CFSR output, so we 
wanted to ensure we properly accounted for this. Ultimately, we obtained and 
evaluated CFSR data for the variables listed in Table 1 at our specified closest 
grid point, as well as for 1° by 1°, and 3° by 3° regions surrounding our central 


location (Figure 10). 
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Figure 10. The three regions for which we obtained CFSR data at 0.5° horizontal 
resolution. The three regions are centered on a CFSR grid point that is 
very close to the average launch site and span a single grid point, a 1.0°x 
1.0° box (yellow), and a 3.0°x 3.0° box (red), 


The CFSR data we used was at 37 vertical pressure levels ranging 
from 1000 hPa to 1 hPa, with a varying vertical separation between the levels. 
Pressure surfaces in the bottom portion of the troposphere (1000 — 750 hPa) had 
a separation of 25 hPa, while the middle to upper troposphere (750 — 250 hPa) 
had a separation of 50 hPa. The separation for pressure surfaces near the top 
varied from a maximum separation of 25 hPa near the 250 hPa pressure surface, 
to a minimum separation of 1 hPa for the highest few pressure surfaces. 


One other consideration we applied in assessing potential data sets 
was their spatial and temporal resolution. One of our main goals was to develop 
sub-daily climatologies for variables that have substantial spatial and temporal 
variability --- due, in particular, to the significant microscale and mesoscale 
variability in coastal environments and clouds. Thus, we needed to work with 


data sets for those variables that have high spatial and temporal resolution, so 
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the data from any individual point in space would be as representative as 
possible of our location of interest (e.g., the average launch site). 


Cc. Cloud-Related Variables 


As noted in Table 1, CFSR cloud level information is given for 
constant pressure surfaces. Rather than considering clouds as one entire 
vertically continuous layer, CFSR breaks the cloud information into layers placed 
into three categories: low, middle, and high clouds. This is similar to the 
categorization of clouds used in standard weather observations made by weather 
personnel around the globe (Saha et al. 2010). The CFSR interfaces between 
the three layers are 650 hPa and 350 hPa for grid points within the bounds of 
45°N and 45°S latitudes. Thus, CFSR low clouds will have bases and tops 
contained below 650 hPa, mid clouds will have bases and tops between 650 hPa 
to 350 hPa, and high clouds will have bases and tops above 350 hPa (Figure 
11); 


High Clouds 


Middle Clouds 


650 hPa 


Low Clouds 





Figure 11. Visual depiction of how the three CFSR cloud layers are defined. This 
depiction is only valid for locations between 45°N and 45°S latitudes. 
Pressure levels and red boxed areas represent the interfaces between the 
layers. 
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2. Meteorological Aerodrome Report (METAR) 


We used METAR data from the KSC Shuttle Landing Facility (SLF), 
located approximately 2.5 nm from the coastal launch sites in the heart of KSC 
(Figure 12). Human observations have been made at the SLF location since the 
inception of the space program. The instrumentation at this site has varied 
through the years, but, until 2011, a human observer was always a part of the 
process. Having a human observer is beneficial to ensuring the accuracy and 


timeliness of meteorological measurements. 
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5 nm From Average Launch Site 
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Average Launch Site: 
24 Latitude: 28.5592°N; 
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Image from: http://www.gpsvisualizer.com/calculators 


Figure 12. Map indicating location of the official Shuttle Landing Facility (SLF) 
METAR observing site. This is the location of a human observer and 
meteorological equipment used to measure cloud conditions. Image 

created from GPS Visualizer [accessed online at 


http:/Awww.gpsvisualizer.com/calculators, Sept 2011]. 


METARs taken in North America follow the World Meteorological 
Organization (WMO) standards, with only a few modifications for units of 
measure. A METAR provides many details of a surface weather observation, but 
for the purposes of this study, the key information needed concerned clouds. 
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The main cloud data included in a METAR observation is the height of the cloud 
base, which is reported in feet for the SLF. WMO standards dictate that 
METARs are taken a minimum of once per hour on the hour, and at other times 
when certain meteorological conditions are met (known as special observations). 
We obtained the extensive set of METARs for the SLF from the climate systems 
division of the 14th Weather Squadron (14 WS) in Ashville, NC. 


3. RAOBs 


Standard weather balloon launches occur approximately twice per day 
(00Z and 12Z) from the CCAFS weather balloon facility. Various types of 
balloons and sounding equipment are released from this location to investigate 
the atmospheric conditions above the region. The CCAFS weather balloon 
facility is unique compared to other balloon facilities because the RAOB launch 
times vary based on meteorological and operational needs. During rocket launch 
operations, weather balloons are released more frequently; as many as ten 
within a six hour window. During summer months, sea breeze activity in the 
region generates a greater threat for severe weather to occur; therefore, balloon 
release times may vary to accommodate 45 WS members investigating the 
severe weather potential As with METARs, RAOBs provide several 
meteorological parameters, but for this study, we only needed the temperature 


and dew point data. 


The 14 WS provided archives of the RAOB data for CCAFS (WMO 
identifier KXMR). Additionally, the 14 WS processed the RAOB data through an 
algorithm based in part on the hypsometric equation to interpolate the data to the 
nearest 500 ft level, so that the data is provided at 500 ft intervals. The raw 
RAOB data is available at a vertical interval of approximately .25 km, or 
approximately 820 ft. Therefore, the 500 ft interpolated RAOB data has a slightly 
enhanced vertical resolution over the raw sounding data, and therefore increases 
the ability to describe the vertical structure of the temperature and dew point 
data. 
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The interpolated RAOB data also had the advantage for our study of being 
available in even 500 ft increments, rather than in the uneven increments of the 


raw sounding data. 


4. Expert Meteorologist Input 


For our investigation of CFSR, METAR, and ROAB data, we developed a 
process for comparing the results from these data sets to actual cloud conditions 
at and near CCAFS/KSC. One way we did this was by comparing satellite 
imagery to visualizations of the data sets for specific case study. Another 
method was to survey meteorologists with extensive experience in the 
CCAFS/KSC region on cloud conditions throughout the year. Our main 
application of this method was for determining the approximate heights of cloud 
tops. In particular, we collected information from these meteorologists on cloud 
top heights for different cloud types and for each month. This was a collaborative 
effort with a team of expert meteorologists from the 45 WS and the Naval 
Postgraduate School. Collectively, this included input from a panel of five 
meteorologists who have an average of 23 years of operational weather 
forecasting and observing experience. All have first-hand experience with 
operational weather support to the space program at CCAFS/KSC, including the 
evaluation of LLCCs during launch countdowns. In developing their inputs to our 
research project, these meteorologists were able to draw on information not 
electronically available to other personnel --- for example, weather aircraft 
reports, weather radar, and weather satellite data related to launch operations 
that is only maintained in hard copy format. The inputs of these expert 
meteorologists was crucial to putting together the climatological information we 
needed to evaluate and supplement the CFSR, METAR, and RAOB data sets, 
and to develop the probabilities of violations of the thick cloud LLCC. 


5. Summary of Data Set Selection 


We selected the CFSR, METAR, RAOB, and expert meteorologist inputs 
as our main potential data sources. However, no one of these data sources had 
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all the information we needed. But the combination of CFSR, METAR, RAOB, 
and expert meteorologist input data provided information that came very close to 
what we needed. For example, these four sources all provided data within 


approximately 2.5 to 3 nm of the average launch site (Figure 13). 
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eee - 


Average Launch Site: 
Latitude: 28.5592°N; 





Image from: http:/Awww.gpsvisualizer.com/calculators 


Figure 13. Map indicating locations of the CFSR, METAR, and RAOB data sources. 
Expanded green box represents the details of the grid points for CFSR. 
Image created from GPS Visualizer [accessed online at 


http:/Awww.gpsvisualizer.com/calculators, Sept 2011]. 


Our initial selection of data from the first three sources amounted to 
hundreds of thousands of lines of data, and required over 2 GB of storage space. 
To increase the efficiency of our data management and analyses, we determined 
that condensing the data set inventory would be necessary. In addition, we 
wanted to condense the data set to help focus our efforts on evaluating the data 
and determining the confidence we could have in the information provided by 
each data set. Thus, we developed a reduced data set that is summarized 


below. 
1) CFSR Data 


a. Single grid point only: 28.5N ; 80.5W) 
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b. 00Z and 12Z only 
c. 1 Jan 1988-31 Dec 2010 
2) METAR 
a. 00Z and 12Z only 
b. 1 Jan 1988-31 Dec 2010 
3) RAOB 
a. 00Z and 12Z only 
b. 1 Jan 1988-31 Dec 2010 
4) Expert Meteorologist Input 
a. Monthly averages for 00Z and 12Z only 


Note that all the data for this study are from the CFSR, METAR, RAOB, 
and expert meteorologist data sets, and are for 00Z and 12Z for the period 1 
January 1988 through 31 December. Thus, our climatological probabilities of 
violation and our other results are based on data for these times and dates. 


One complication in using these data sets is that there were some 
temporal inconsistencies between the data sets. For example, CFSR provided 
data for every variable for OOZ and 12Z for every day during January 1988-2010. 
However, there were times for which 00Z or 12Z METAR data was not included 
in the archive. There were also periods in which both a special METAR and 
standard hourly METAR report made it into the archive and were valid at the 
same time. As noted previously, RAOB release times from CCAFS varied, 
which resulted in many of the dates during our study period having no 00Z and/or 
12Z observations. To compensate for this, we assumed any RAOB valid from 
10Z to 14Z to be representative of 12Z, and counted it as a 12Z observation. 
Similarly, we assumed any RAOB valid from 22Z to 02Z to be representative of 
00Z and counted it as a 00Z observation. Table 2 summarizes the availability of 
data from the CFSR, METAR, and RAOB data sets. 
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Total Observations 
CFSR |METAR| RAOB 


| 002 | 11680 | 12076 | 4370 
| 122 | 11680 | 12445 | 9911 





Table 2. Total number of times for which data set was available and used in our 
study from the three data sources shown for 00Z and 12Z of our study 
period of 1 January 1988-31 Dec 2010. 
E; METHODS 
le Overview 
We designed our research methods to deliver the following main products: 
1. Climatological thick cloud LLCC violation data set 
2. Climatological probabilities of violating (POV) thick cloud LLCC data set 
3. Data set describing thick cloud POV for a range of thickness thresholds 


We generated all three of these products. The second and third products were 
derived from the first. The third product was the output from our sensitivity 


analyses. 


Our overall approach was to process the data sets described in the prior 


section to determine the following variables. 
1) Determine cloud base height 
2) Determine cloud top height 
3) Calculate cloud thickness 
4) Determine heights of 0°, -15°, and -20° Celsius levels 


Once we determined these four variables, we used them to compute the 
products listed above. Each data set we worked with has its own limitations, and 
not all of them offered every variable we needed. For example, the METAR data 


set only gave information about cloud bases. The following sections outline the 
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how we used each data set individually (CFSR, METAR, RAOB, and Expert 
Meteorologist), how we used data from one data set to test the data from another 
data set, and how we combined data from the different data sets to develop a 


merged data set. 


2. CFSR 


We investigated CFSR to determine cloud base, top, and thickness 
information. This included gathering information for both heights and 
temperatures at the cloud bases and tops. As listed in Table 1, the CFSR data 
we processed contained cloud base and top information relative to given 
pressure surfaces. The thick cloud rule is evaluated for a thickness requirement 
measured in feet. Therefore, we developed a process to convert CFSR cloud 
bases, tops, and thicknesses from pressure surfaces to feet. To do this, we 


made use of the hypsometric equation. 
Thickness = Z2 — Z, = Int 
Io 


The specific derivations and details of the hypsometric equation can be 
located in numerous meteorological texts, but we referenced Wallace and Hobbs 
(1977), and Holton (2004). For the purposes of this study, and based on the 
format of the CFSR data, we needed to make some assumptions and 
modifications to the use of the hypsometric equation. Listed below are the 
details on our use of the hypsometric equation for determining the thickness of a 


cloud layer. 


e Z,-—Z;, Is the thickness of the layer (in meters), where Zzis the 
highest point in altitude of the layer, and Z,is the lowest point in 
altitude of the layer. 


e Ris the gas constant for dry air (287 ak ) 


e T is the average temperature (in Kelvin) of the layer 
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e is the global average gravitational acceleration at mean sea 


level (9.80665 —) 


e is the pressure (in hPa) of the layer’s lower surface 
° is the pressure (in hPa) of the layer’s upper surface 


Table 3 summarizes the variables, or parameters, we used calculate cloud 
layer thickness using the hypsometric equation. Note that CFSR provided data 
for all the variables needed for this calculation, except T at the cloud base. 
However, CFSR does contain temperature values at the 700-350 hPa pressure 
levels in 50 hPa increments. We used this information to make several 


modifications and assumptions to our application of the hypsometric equation, so 


that we could calculate cloud layer thicknesses. 
















Parameter 
Geopotential Height (m) Pressure (hPa) Temperature (K) |Total Cloud Cover (%) 


Entire Atmosphere 
Low Cloud Level 
Middle Cloud Level 


High Cloud Level 



























—-OoO < oO FT 







—-no-e70< 


Level of 0° Isotherm 











Table 3. |The CFSR variables we used in our thick cloud LLCC study. The levels 


listed in each of the four parameter columns indicates the vertical levels 
for which we obtained and used the CFSR data for that parameter. The 
yellow highlighting indicates the parameters and the levels for which 
CFSR data was used to calculate cloud layer thickness using the 
hypsometric equation. 


a. Average Temperature (T) Calculations 


To calculate the average temperature, T, we used two temperature 
levels; cloud base, and cloud top. For purposes of this section only, Ty is the 
temperature at cloud base, T; is the temperature at cloud top, and Ty is the 
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temperature at the indicated pressure surface, where p is the pressure surface in 
hPa. The average temperature was calculated as: 


T,+Tp 
2 


T= 


As previously noted, anytime the model indicated a cloud, the 
CFSR output would contain the temperature at cloud top. However, the 
temperature at cloud base was not a given value. To calculate the temperature 
at cloud base, we made the assumptions and modifications summarized below. 
Additionally, some of the calculations below were dictated by the fact that our 
CFSR data set contained information between 700 and 350 hPa only. The 
process we used to determine these temperatures is summarized in the 


equations below and in Figure 14. 


1) For clouds with bases and tops below (at an altitude lower 
than) the 700 hPa level: 


Tp =T; 


2) For clouds with bases below, but tops above (at an altitude 
higher than) 700 hPa: 


Ty = T700 


3) For clouds with bases and tops above 700 hPa, but below 
350 hPa: 


T, =T + (P below cloud bottom—P cloud bottom) . 
b — * p below cloud bottom 50 


(Tp above cloud base — Ty below cloud pase) 
4) For clouds with bases and tops above 350 hPa: 


Tp = T350 
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Cloud top temperature (7,) Known from CFSR 


a“ 





Figure 14. Visual representation of an example of how the P;, Po, T;, and Tp surfaces 
were evaluated and applied to the hypsometric equation to calculate the 
cloud thickness. 


In the example from Figure 14, the cloud base was located 
somewhere between the 650 and 600 hPa pressure surfaces. CFSR provided a 
pressure value in hPa for the cloud base. The estimated cloud base temperature 
was computed based on using the temperatures at the pressure surfaces above 
and below the cloud base in the manner described below. 


(Pcloud base IS the CFSR given pressure at cloud base) 


36 


Recall that the CFSR cloud data is allocated in 50 hPa increments, which 
is why we chose to divide our pressure surfaces into 50 hPa increments in the 


equation above. 


b. Cloud Thickness Calculations 


Once we were able to calculate the average temperature of the 
cloud layer, we could then apply the full use of the hypsometric equation. The 
thickness of any given cloud layer in CFSR was computed with the set of 
variables listed in Table 1, and using the average temperature calculations listed 
in the previous section of this study. Using the hypsometric equation yielded a 
thickness measurement in meters, so we also needed to apply a final conversion 
from meters to feet. We used the standard conversion of 1 meter = 3.2808399 


feet. Therefore, our final CFSR cloud thickness equation was calculated as: 


R:T 





Py 
‘In =) -3.2808399 


Thickness (ft) = Z2 —Z, = ( 
Jo 2 


Cc. Cloud Base Height and Cloud Top Height Calculations 


After calculating cloud thicknesses, we then solved the hypsometric 
equation for the estimated heights of the cloud tops (Z,), and cloud bases (Z;). 
Recall that the initial CFSR data set included heights of pressure surfaces in 50 
hPa increments between 700 and 350 hPa. We used this information to define 
the cloud base and cloud top heights in meters, and then converted to feet. The 
CFSR cloud base pressure was located between two of the defined pressure 
surfaces. For example, if CFSR gave a cloud base pressure of 610 hPa, the 
cloud base pressure would be located between the 650 and 600 hPa pressure 
surfaces (Figure 15). Using the hypsometric equation, we calculated the 
thickness of the layer between the pressure surface below the cloud base and 
the pressure at the cloud base. 
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350 hPa 


500 hPa 


Pressure at 
cloud base 
given by 
CFSR 


550 hPa 


600 hPa 


650 hPa 


Height of 
pressure 
surfaces 
given by 
CFSR 





Figure 15. Visual representation of an example of how the P;, and Pz surfaces were 
used in calculating the height of the cloud base. Pzis the pressure given 
by CFSR at the base of the cloud, and P; is the height of the first CFSR 

given pressure surface located below the base of the indicated cloud. 


Once the thickness of this layer was determined, we applied the 
known height of the pressure surface located below the cloud base to solve the 
hypsometric equation for the cloud base. 


e Original hypsometric equation: —_- — 


e Re-arranged for the cloud base: —- — 
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This method was applied using a similar definition of the variables similar to that 
in the previous application of the hypsometric equation, but with the modifications 
listed below. 


e Z, is the height of the cloud base 


e Z,is the height of the closest pressure surface below the cloud base 
given by CFSR 


e P, is the closest pressure surface below the cloud base 


e P, is the pressure at the cloud base 


Temperature at P7+Temperature at P, 
2 


e T is calculated as 


To calculate the cloud top heights, we simply added the cloud thickness to 
the cloud base height. Since both the cloud base height, and thickness were 


already converted to feet, no further conversion was necessary. 


d. Height of 0° Isotherm 


The only temperature level directly output via CFSR was the height 
of the 0° isotherm, in meters above Earth’s surface. The only modification we 
needed to make was to re-calculate the values to be expressed in feet. Again, 
we used the standard conversion of 1 meter = 3.2808399 feet. Multiplying every 
CFSR output of the 0° isotherm height by this value satisfied the requirement. 


3. METAR 


As previously stated, only the METAR cloud base information was used 
for this study. For every observation in our data set, we simply extracted the 
cloud base height and set every other part of the METAR observation aside in a 
separate file. Cloud base information is reported in a METAR as the amount of 
sky they are estimated to cover, with the cloud base heights measured in feet. 
Coverage is estimated in 1/8ths of the sky (oktas), and is based on summing 
upward from the surface to the highest layer of cloud observed. Therefore, a 


weather observer, or cloud measuring equipment, detects and reports clouds 
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from the ground up. This indicates the potential for an overcast layer of lower 
clouds to obscure a higher layer of clouds from the view of the observer or 
equipment. WMO standards dictate cloud heights be reported by level. There 
are three cloud layer groups, or what WMO calls genera: low, mid, and high 
cloud layers (WMO 2008). The average base height of low, mid, and high clouds 
varies by latitude, but our study region fell within the WMO standards for 
temperate regions. Tables 4 and 5 summarize the cloud layer height ranges, as 
well as reporting increments. We assigned this cloud layer definition to all data 


sets. 


Cloud Layer Base Height Ranges (ft) 
Low Cloud Mid Cloud High Cloud 





6,500 


Table 4. Summary of defined cloud base heights for temperate regions. Data 
taken from World Meteorological Organization Chapter 15 “Observation of 
Clouds” (accessed online at http://library.wmo.int) 


Increments of Reportable Values of Cloud Base Height 


Range of Height Values (ft) | Reportable Increment (ft) 


< 5,000 To nearest 100 
> 5,000 but < 10,000 To nearest 500 
> 10,000 To nearset 1,000 





Table 5. Summary of reportable cloud base height values. Data taken from 
Federal Meteorological Handout—1 Chapter 9 “Sky Condition’. 


For the purposes of this study, we primarily needed to work with METAR 
information on cloud base heights and whether clouds were reported or not. For 
this reason, we separated the cloud coverage amount from the base height, 
which resulted in a data set containing only METAR cloud base heights for all 
00Z and 12Z times. 
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3. RAOB 
a. Cloud Bases and Tops 


RAOB data yields information relative to the vertical structure of the 
atmosphere. There is no definitive way to extract cloud layer information from 
RAOB data; however, there are empirical rules that are widely accepted for basic 
applications. The basic principle involves using the RAOB temperatures and 
dew point depressions to calculate relative humidities. The Air Force Weather 
Agency (AFWA) document, “Meteorological Techniques’(AFWA/TN-98/002, 
2006) contains information to aid operational forecasters with techniques to 
improve forecasting skills. For this study, we used the cloud layer temperature 


and dew point depression rules set in “Meteorological Techniques” to identify 


cloud layers from RAOB data. 


Cloud Base Temperature / Dew Point Depression Relationship 





Table 6. Summary of RAOB temperature and dew point depression relations and 
how they were applied in thick cloud LLCC study. 


RAOB data was given in 500 ft increments from the surface up to 
sounding completion, which can vary for every RAOB. However, based on the 
thick cloud LLCC temperature thresholds, we only used RAOB data from the 
surface up to the -20°C level. Therefore, we did not calculate RAOB cloud data 
at temperatures colder than -20°C. Each line of RAOB data contained 
temperature and dew point values at the specified time. To compute the dew 
point depression, we simply subtracted the dew point from the temperature. We 
evaluated RAOB data for clouds by beginning at the lowest level, and worked up 
in the vertical to sounding completion, or -20° C level, whichever occurred first. 
As we followed the RAOB in the vertical, the first level in which one of the 


requirements defined in Table 6 was met was classified as the cloud base. 
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Continuing in the vertical, we continued to identify a cloud layer until we reached 
a level at which the requirements defined in Table 6 were no longer met, which 
we defined as the cloud top. 


As soon as a weather balloon is released, it will ascend and drift 
horizontally, based on the ambient atmospheric conditions. However, for most 
applications in our study, we assumed the entire sounding was representative of 
the vertical profile of temperature and dew point information directly over the 


study region. 


b. Height of 0°C, -15°C, and -20°C Levels 


The heights of the 0°, -15°, and -20°C levels were derived using the 
AMU RRA utility tool with the RAOB data set. All the raw temperature values 
were given in degrees Celsius, and heights were given in feet, therefore no 


further conversions were necessary. 


4. Expert Meteorologist Input 


We collected expert meteorologist information only on cloud top heights. 
Using their decades of experience, each expert provided their best approximation 
of the monthly average cloud top heights by layer (low, mid, and high). Once the 
experts submitted their inputs, we weighted each input equally and averaged 
them to determine an overall expert estimate of the cloud top heights. The 
formula for computing the expert meteorologist estimate is shown below. The 
results from applying this formula to the expert inputs are shown in Table 7. 


Expert 1+Expert 2+Expert 3+Expert 4+Expert 5 


Monthly Average = = 
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Monthly Average Cloud Top Heights 
Based on Expert Inputs 


6500 


18000 





Table 7. Monthly average cloud top heights (in feet) for the three cloud layers, low, 
mid, and high. Values based on averaging the inputs from five expert 
meteorologists. 


5. Data Quality Control 


Since our process involves the application of multiple data sets, 
developing a quality control method was critically important. We divided our 


quality control efforts into four categories: 
1. Cloud detection 
2. Cloud bases 
3. Cloud tops 
4. Temperatures 


For each of these categories and for each data set, we compared from 
one data to that from the others and to other meteorological information. Our 


objective was to identify the similarities and differences between each data set, 
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and to assess the accuracy of each data set. Ultimately, we wanted to determine 
the confidence to place in specific parts of each data set, and thereby determine 
if and how to apply those parts in the calculation of thick cloud LLCC 
climatologies. The following sections describe the processes and methods we 
developed and applied to the data sets in our quality control procedures. Unless 
otherwise noted, all data set comparisons were performed for 00Z and 12Z of 
January 1988 — Dec 2010. The specific results of these data set comparisons 


will be discussed in Chapter III. 


a. Cloud Detection 


We examined and compared the detection of clouds by the CFSR, 
METAR, and RAOB data sets. We defined cloud detection as the identification 
of any amount of cloud at any layer, time, or date. If no clouds were reported, 
then clear skies were identified. We calculated and compared the percentage of 
times that clouds and clear skies were detected by CFSR, METAR, and RAOB. 
We used percentages rather than total numbers because the number of times 
within the study period that data was available was different for each data set 
(see Table 2). For these comparisons, we assumed the METAR observations 
were a close representation of the real state of cloudy or clear sky conditions 
over the study region, and therefore used those observations as our ground truth. 
We compared both cloud and clear sky conditions. For each data set (CFSR, 
METAR, and RAOB), we applied the formula below (or a similar one for cloudy 
conditions. We applied this formula separately for OOZ data and 12Z data. 


Total number of times with clear skies reported 100 


Total number of observations 


In this formula, the number of number of times is the number of 00Z 


or 12Z occasions during the study period (Table 2). 


b. Cloud Base Heights 


Cloud base height comparisons were conducted using monthly 


averages for the entire data base. We compared CFSR, METAR, and RAOB 
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derived cloud base heights, separated by layer (low, mid, and high). For this 
study, we defined cloud layers using the information in Table 4. As with cloud 
detection, we assumed METAR observations to be a close representation of the 
actual cloud bases, and used them as our ground truth. Based on our 
procedures for identifying cloud bases in each data set (see prior sections), we 
calculated the cloud bases for each day in our data base, and then calculated the 
averages for each month. These monthly averages are what we used to make 


our final comparisons of the data sets. 


Cc. Cloud Top Heights 


Cloud top height comparisons were conducted using the same 
method as for the cloud bases, but with the one exception. METAR observations 
do not contain information on cloud top heights, so we used expert meteorologist 
Input as our ground truth for cloud top heights. The cloud top comparisons were 
conducted in the same way as for the cloud base comparisons. 


d. Temperatures 


Only two of our data sets included vertical profile temperature 
information, CFSR and RAOB. The only common variable between them was 
the height of the 0° Celsius isotherm. Similar to cloud detection, and cloud base 
comparisons, we wanted to identify a data set we could assume to be a close 
representation of the real state of the vertical temperature profile. For this, we 
assumed the data interpreted from the RAOBs would be an accurate 
measurement of the true heights of temperature levels over our study region. 
Thus, the RAOB data served as our ground truth for temperatures. Our objective 
in this comparison was to determine how much confidence to have in the CFSR 
and RAOB temperatures, so we could determine how to apply them in 
developing climatologies for the thick cloud LLCC. 


45 


6. Hybrid Process 


Upon completion of the data set comparisons, we were able to compile a 
list of the strengths and weaknesses of each data set. Based on these results 
we produced a merged data set based on blending different parts of each of the 
four data sets. CFSR data composed the bulk of this merged data set, so we 
referred to the merged data set as the modified CFSR data set. 


The CFSR was very complete spatially and temporally. But, as we 
discovered, CFSR values for some variables were not as accurate as the values 
from the other data sets (METAR, RAOB, expert meteorologist inputs). On the 
other hand, the METAR, RAOB, and expert meteorologist input information was 
spatially and/or temporally incomplete. Therefore, to create our merged data set, 
we used CFSR as the underlying foundation, and used data from the other data 
sets for the variables for which those other data sets appeared to provide more 


accurate information. 


We began our development of the modified CFSR data set with the cloud 
base heights and cloud top heights. Due to the differences in the total 
observations contained in each data set, our comparisons of cloud base heights 
and cloud top heights were based on monthly average heights. We calculated 
the monthly averages of cloud base heights and cloud top heights for the entire 
study period. We separated the averages by time (00Z and 12Z), and by layer 
(low, mid, and high clouds). For this study, we assumed METAR cloud base 
heights to be the best representation of the real state of cloud base heights, and 
the expert meteorologist input to represent the real state of cloud top heights. 
Based on these assumptions, we applied the monthly average METAR cloud 
base heights to adjust the CFSR cloud bases, and the expert meteorologist input 
cloud top heights to adjust the CFSR cloud top heights. 


These adjustments were made via corrections terms that we calculated by 
subtracting the monthly mean CFSR heights from the monthly mean heights from 
the alternative data set. Thus, for cloud base heights, we subtracted the monthly 
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mean CFSR cloud base height from the monthly mean METAR cloud base 
height; and for the cloud top heights, we subtracted the monthly mean CFSR 
cloud top height from the monthly mean expert meteorologist input cloud top 
heights. We derived these correction terms by layer, by time (00Z and 12Z), and 
by month for cloud bases and cloud tops. This gave us four correction term sets 
for each of the three layers and for each month. Examples of the 00Z sets of 


correction terms are given in Tables 8 and 9. 





Cloud Base Height Correction Terms 


Jan 1988 -Dec 2010 Correction Terms (ft) 
| Month | low | Mid | High | 








uw 
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Table 8. Cloud base correction terms for 00Z calculated as monthly averages in 
feet for each cloud layer (low, mid, high). Terms calculated based on the 
difference: METAR cloud base height — CFSR Cloud base height. 
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Cloud ——— Height Correction Terms 
Jan 1988 -Dec 2010 | Jan 1988-Dec2010 _—‘| Dec 2010 Correction Terms a 


—— ——_ 
-18927 
Feb | 


sek 


Feb -9956|  -1801 


-1040 


-817 


-1853 
-1891 


-564 -2219 


A 
9 

-6803 -1819 
9 
1 


Jul -585 
-592 -2340 


| Jul 
| Aug 
: -2629 
| Oct 


-2203 


O - 2436 


| Nov | -938| -10271| _-2139 


-1992 


-333|  -8273] -2101 


Table 9. Cloud top correction terms for 00Z calculated as monthly averages in feet 
for each cloud layer (low, mid, high). Terms calculated based on the 
difference: Expert meteorologist input cloud top height — CFSR Cloud top 
height. 





This correction terms were then applied to the original CFSR data set 
according to the month being computed. For example, to calculate the low cloud 
base height for 00Z for an individual January day, we subtracted a value of 1,933 
feet (see Table 8) from the original CFSR cloud base height. For February we 
did the same except that we subtracted a value of 2,082 feet. We continued this 
process for each day in the data base for 00Z and 12Z, as well as for each layer. 
Once these correction terms were applied, we were left with a modified set of 
CFSR cloud base heights and top heights that were adjusted toward the METAR 
cloud base heights, and the expert meteorologist input cloud top heights. 


Our comparisons of the RAOB and CFSR temperature profiles for the 
study period and region showed that the two data sets provided very similar 
information. Thus, there was no need to correct the CFSR temperature height 
level data. However, since the CFSR data set only contained the heights of the 
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0° C level, we used the RAOB data to determine the heights of the -15° and -20° 
C level and to supplement the CFSR data. 


Figure 16 outlines the methods we used to determine the cloud base 
height, cloud top height, and temperature height level data that we included in 
our modified CFSR data set. 


Modified CFSR 


Expert 
Meteorologist 


F Clou 
foal 


= Temperature levels 





Figure 16. Schematic of the processes used to develop the cloud base height, cloud 
top height, and cloud temperature height level data included in the 
modified CFSR data set. Data flows into CFSR represent the use of non- 
CFSR data to evaluate CFSR and to adjust and/or supplement CFSR 
data. METAR data were used to modify CFSR cloud base heights. 
Expert meteorological input was used to modify CFSR cloud top heights. 
RAOB data were used to confirm and supplement CFSR temperature 
height levels. 
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7. 


Climatology and Probability 
a. Initial Climatology and Probability 


Once the correction terms were applied, and we constructed the 


modified CFSR data set. We then used that data set to calculate cloud base 


height, cloud top height, cloud thickness (using the hypsometric equation as 


described above), and the heights of the 0°, -15°, and -20°C levels for each cloud 


layer and for 00Z and 12Z of each date in the entire study period. 


The thick cloud LLCC is complex due to the varying temperature 


constraints applied to the general violation of 4,500 ft thickness. In determining 


our final climatological probabilities, we used a series of logical tests for each 


time, date, and cloud layers. We determined a violation of the thick cloud LLCC 


had occurred if one of the conditions below was met. 


1. 


Low cloud thickness was greater than or equal to 4,500 ft, and low 


cloud top height was higher than or at the height of the 0°C level 


Mid cloud thickness was greater than or equal to 4,500 ft, and the 
mid cloud top was located between or at the heights of the 0° and - 


20° levels 


Mid cloud thickness was greater than or equal to 4,500 ft, and the 
mid cloud base height was higher than or at the height of the 0°C 
level, but lower than or at the height of the -20°C level 


High cloud thickness was greater than or equal to 4,500 ft and the 
high cloud base height was located between or at the heights of the 
0° and -20°C levels 


If none of these conditions was met, then we determined that a 


violation of the thick cloud rule had not occurred for that time, date, and cloud 


layer. Additionally, once these logical tests were applied, we needed to account 


for the exceptions to the thick cloud LLCC. To accomplish this we applied 


50 


another series of logical tests to any indicated violation. We discounted a 
violation if any of the conditions below were satisfied. 


1. Mid cloud thickness was greater than or equal to 4,500ft, and the 


cloud base height was higher than the height of the -20°C level 


2. High cloud thickness was greater than or equal to 4,500ft, and the 
high cloud base height was higher than the height of the -15°C level 


3. High cloud thickness was greater than or equal to 4,500ft, and the 


high cloud base height was higher than the height of the -20°C level 


To calculate our final climatological probability of violation for each 
day at 00 and 12Z, we counted any violation at any layer as a violation for that 
date and time. We applied a final logical test to each observation time, and 
counted a violation for the date if any combination of the three layers indicated a 
violation. Once completed, our data set revealed a tally of violations by date and 
time for the entire study period. Our total data set accounted for twenty-three 
years of data, which implies twenty-three possibilities for each day of the year to 
violate the thick cloud LLCC for each time (00Z and 12Z). Our final calculation of 
the probability of violation (POV) of the thick cloud rule for any day of the year at 
either 00Z or 12Z is shown in the equation below. The process we used to 
generate the thick cloud LLCC climatologies and POVs is summarized in Figure 
17. 


Total number of violations 


POV = a 


100 
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Figure 17. Schematic of the processes we used to: (1) develop the cloud base 
height, cloud top height, and cloud temperature height level data included 
in the modified CFSR data set; and (2) use that data set to calculate the 
thick cloud LLCC climatologies and probabilities of violations. See the 
Figure 16 caption for additional information. 


b. Final POVs 


We wanted to make the products from our research as 
operationally relevant and useful as possible, so we needed to fine-tune the final 
results. We determined that we could make our climatology products more 
useful by temporally smoothing them to reduce large day-to-day variations which 
appeared in our raw climatologies based on our 23 year long data set. This is 
the same approach Muller (2010) used in his work with natural lightning LLCC 
climatologies. Once the initial POVs were calculated, we applied a centered 
running mean smoother to them. We used five, seven, nine, eleven, thirteen, 
and fifteen day running means for each day of the year for both 00Z and 12Z. 
We then compared the smoothed results and settled on the fifteen day running 


mean results as the most operationally useful. 
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Cc. Sensitivity Analysis 


We conducted a sensitivity analysis to assess the impacts of using 
different thickness thresholds, but with the same temperature constraints. To do 
so, we repeated the process for determining the POVs from the CFSR data set, 
but for a range of cloud thickness thresholds --- in particular, cloud thickness 
thresholds of 3,500, 4,000, 5,000, 5,500, 6,000, 6,500, 7,000, and 7,500 ft. 


d. Case Studies 


We conducted a small set of case studies of known thick cloud 
LLCC violations from 2005-2010. Our purpose was to do a sample validation of 
our POVs against known violations. As described in Chapter II, Section B.1, the 
LWOs identify thick cloud violations using several sources of meteorological 
input, including weather reconnaissance aircraft. Much of this information is 
contained only in written form in the LWOs mission folders, which are kept by 
each individual LWO for each launch mission. A known violation is defined as 
occurring if at any time during a launch countdown the launch weather team 
determined the thick cloud LLCC was violated. 


| Year | Month | Day _ 
November 





Table 10. Dates of known thick cloud LLCC violations. We used the dates of these 
violations to determine whether our calculated thick cloud LLC climatology 
data set also showed thick cloud violation on those dates. 
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We selected eight different missions during 2005-2010 for which 
LWO logs indicated the thick cloud rule had been violated (Table 10). For our 
study, we were only concerned with determining if the thick cloud LLCC was 
violated, and not with how long the violation lasted. We then determined whether 
our calculated thick cloud LLCC climatology data set showed violations at the 
same times and dates as the eight known violations. If our data set showed a 
violation at either OOZ or 12Z of a known violation date, then we determined our 


data set had correctly identified the known violation for that date. 
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lll. RESULTS 


A. DATA SET COMPARISONS 
1. Overview 


We evaluated the different data sets, and compared them to each other, to 
determine which components of each data set to include in our modified CFSR 
data set. As noted in Chapter Il, we conducted our data set comparisons 
analyses, and our climatology data set development, for 00Z, and 12Z. The 
results from these two times were very similar to each other. The 00Z results are 
shown in Chapter Ill and the 12Z results are shown in Appendices A and C. 


2. Cloud Detection 


Using the methods described in Chapter II, Section E.5.a, we compared 
the cloud detections from CFSR, RAOB, and METAR. Figures 18-21 show the 
percentage of 00Z times during the study period when clouds and clear skies 
were detected by the CFSR, METAR, and RAOB data sets. The CFSR and 
METAR data sets both reported clouds 92% of the time, but the RAOB data set 
reported clouds only 28% of the time. As previously stated, since METAR 
surface weather data are based on a combination of instrument measurements 
and human observations, we determined that METAR observations were the 
most representative of the true state of cloudy and clear sky conditions over 
CCAFS and KSC. Thus, Figures 18-20 provide evidence that the CFSR data set 
did well at identifying cloudy conditions and clear skies, while the RAOB data set 


did not, with 64% fewer cloud detections than the other two data sets. 
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Figure 18. Percentage of all OOZ times in the study period for which CFSR indicated 
clouds (blue) and clear skies (yellow). 
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Figure 19. Percentage of all 00Z times in the study period for which METAR data 
indicated clouds (blue) and clear skies (yellow). 
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Figure 20. Percentage of all 00Z times in the study period for which RAOB data 
indicated clouds (blue) and clear skies (yellow). 
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Figure 21. Percentage of all OOZ times in the study period for which the CFSR, 
METAR, and RAOB data sets indicated clear skies. 


There are several possible reasons for the RAOB results. Recall from 
Chapter Il, Section D.5, that the total number of times for which data was 
available were different for each data set, with the RAOB data set having a much 
lower number than the CFSR and METAR data sets. Additionally, we inferred 
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cloud occurrence from the RAOB data based on empirical rules using dew point 
depression and temperature relationships. This method could have led to 
missing days and times when clouds actually were present, but the RAOB data 
did not show those relationships. Additionally, days when the only clouds 
observed were in the mid and high cloud layers might have been missed by the 
RAOB due to wind drift of the RAOB away from the CCAFS and KSC location. 
This RAOB drift means the sensor could be tens of miles downstream of the 
study region by the time it reaches the heights of the mid and high cloud layers. 
This could lead to a RAOB sounding not detecting mid and high clouds over 
CCAFS or KSC, either missing them altogether, or observing clouds over an 
entirely different region. 


Once the initial cloud detection comparisons were complete, we decided 
to look deeper into the relationship between CFSR and METAR to ensure they 
displayed similar characteristics. We conducted a comparison of cloud detection 
for all years in the study period for January and July only. We calculated the 
percentages of cloud detection for all January detections in the study period, and 
repeated the process for July. We selected January and July because are they 
are the climatological extreme months, with cloudy days being close to a 
minimum for the year in January and a maximum for the year in July, according 
to the 45 WS Forecast Reference Notebook (FRN). Both CFSR and METAR 
demonstrated what we expected in the January and July comparisons, with 
cloudy and clear sky percentages that were very similar to each other (Figure 
22). 


Overall, our cloud detection results provide evidence that CFSR data did 
well at distinguishing cloudy days from clear sky days, while the RAOB data did 
not perform as well, assuming that the METAR data is representative of the true 
cloud conditions over the study region. 
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Figure 22. Percentage of all OOZ times in the study period for which the CFSR data 
set (top panels) and the METAR data set (bottom panels) indicated clouds 
(blue) and clear skies (yellow) for January (left panels) and July (right 
panels). 


3. Cloud Base Heights 


As with cloud detection, we treated the METAR data as our best indicator 
of the true cloud base heights over our study region. We used the processes 
outlined in Chapter II, Section 6.6, and the WMO cloud layer definitions (Table 4) 
to compare the cloud base heights from the three data sets (CFSR, METAR, and 
RAOB). Our objective was to determine which set to apply in developing our 


thick cloud climatology data set. 


Figure 23 shows that the METAR and RAOB low cloud base heights were 
in generally good agreement, while the CFSR low cloud bases were significantly 
higher. The differences between CFSR and the other data sets ranged from 
approximately 2,000—3,000 feet in the winter, to 4,000—5,000 feet in the summer. 
Additionally, CFSR indicated a significant seasonal variation in cloud base 
heights, with an increase of nearly 3,000 feet in the low cloud base heights from 
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January to July. In contrast, the METAR data indicated a decrease of low cloud 
base heights of approximately 500 feet from winter to summer. 


00Z Low Cloud Base Comparisons 
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Figure 23. Monthly average low cloud base heights for each data set from 1988- 
2010. 


The CFSR mid cloud base heights were between 7,000 and 9,000 feet 
higher than those from the METAR data (Figure 24). The RAOB mid cloud base 
heights were also higher than those from the METAR data by about 2,000 to 
3,000 feet. For all three data sets, the seasonal variation in mid cloud base 
heights was small 
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00Z Mid Cloud Base Comparisons 
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Figure 24. Monthly average mid cloud base heights for each data set from 1988-— 
2010. 


CFSR high cloud base heights were 13,000 to 20,000 feet higher than 
those from the METAR data, and those from the RAOB data were about 4,000 to 
5,000 feet lower than those from the METAR data (Figure 25). Note in Figure 25 
the lack of RAOB high cloud base heights for January through March. This is 
because our RAOB data set only contained information up to the height of the - 
20° C level. This limitation in our RAOB data set arose from our RAOB data 
request being based on the thick cloud LLCC -20° C temperature threshold. In 
January-March, the high cloud base heights tended to occur at temperatures 
colder than -20° C, so we did not capture these clouds in our RAOB data set. To 
correct for this limitation in our RAOB data, we used linear interpolation to 
estimate the January-March values. We selected this method was based in large 
part on the relatively constant RAOB high cloud base heights in April-December 
(Figure 25). The interpolation yielded an average value for January-March of 
20,500 feet, very similar to the mean for the rest of the year of 21,000 feet. 
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Figure 25. Monthly average high cloud base heights for each data set from 1988-— 
2010. 


To assess interannual variability in the cloud base heights, we conducted 
a comparison of the cloud base heights from each data set for January and July 
for all years in the study period. We did this in part to determine if the high CFSR 
cloud base heights were due to some extreme outlier years. None of the three 
data sets indicated any significant year-to-year variations in any layer for either 


January or July (results not shown). 


We also analyzed variability in the METAR cloud base heights by 
calculating the upper and lower tercile limits of the monthly cloud base heights for 
based on data from all years. The results indicated very small magnitudes of 
difference between the upper and lower terciles for each layer (Table 11). Low 
cloud base heights only had a difference of 900—1700 ft between lower and 
upper terciles, mid clouds had a difference of 3,000-—4000 ft, and high clouds had 
a difference of only 1,000 to 2,000 ft. This indicates relatively low variability in 
the METAR cloud base heights. 
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Jan 1988 -Dec 2010 Upper Tercile 


Total Obs| Low igh Low 


Jan 29768 210 2500 380 
250 390 


900 2500 4 


0 
40 


900 2500 35 


31 


1000 2400 3 


1 

31 
10000| 2400 31 

37 
2500 40 
2500 400 
2466 360 


Table 11. Monthly average lower and upper tercile values for METAR cloud base 
heights in feet for the three cloud layers (low, mid, high). We used these 
values to identify the typical range (middle third) of cloud base heights for 

each cloud layer. 
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Overall, the cloud base height comparisons revealed the CFSR cloud 
base heights were much higher than the METAR and RAOB cloud base heights. 
Additionally, applying experienced meteorological reasoning, much of the CFSR 
cloud bases appeared unrealistically high. For example, the average height of 
CFSR high cloud bases over our study region was 41,717 feet, corresponding to 
an average high cloud base height temperature of -57°C (using temperature 
climatology from RAOB data). The likelihood of cloud formation greatly 
diminishes at temperatures colder than —40°C (Rogers and Yau 1989). Also, the 
degree of separation between CFSR values and METAR values increased 
upward, from the smallest differences for the low cloud layer to the largest 
differences for the high cloud layer. Based on these results, we chose to use the 
METAR data to determine cloud base heights in the development of our thick 


cloud LLCC climatologies. 
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4. Cloud Top Heights 


As defined in Chapter II, Section 6c, we conducted cloud top height 
comparisons using the same process as for cloud bases, except that we did not 
use METAR data and we did use expert meteorologist input. The expert 
meteorologist input is based on numerous years of monitoring varying weather 
conditions near CCAFS and KSC, using first-hand weather aircraft reports, radar, 
and satellite observations. So we regarded the expert cloud top heights for all 


three cloud layers as the most realistic. 


We compared the cloud top height information from each data source 
(CFSR, Expert, RAOB) for each cloud layer. Figure 26 shows that the low cloud 
top heights were lower in winter and higher in summer for each data source. 
This is consistent with the expected thinning and thickening of the troposphere 
during winter and summer. This seasonal variation was largest (smallest) at 
about 8,000 (2,500) feet in the expert (RAOB) low cloud top heights. 


00Z Low Cloud Top Comparisons 
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Figure 26. Monthly average low cloud top heights for each data set from 1988-2010. 
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The CFSR low cloud top heights showed a seasonal cycle that was in 
phase with the corresponding expert heights but with about half the amplitude of 
the expert seasonal cycle. One reason for this smaller seasonal variation may 
be that the CFSR low cloud base heights were too high (Figure 23). Recall that 
our cloud base height comparison showed that CFSR cloud bases for all cloud 
layers were significantly higher than those indicated by METAR data (Figures 23- 
25). Thus, CFSR would be expected to also give cloud top heights that are 
unrealistically high (cloud tops cannot be lower than cloud bases). Note that for 
January-March, the CFSR low cloud top heights were 1,000 — 3,000 feet higher 
than the corresponding expert values. For these months, the CFSR low cloud 
base heights were 2000 - 2500 feet too high (Figure 23). This suggests that this 
may be an example of the CFSR low cloud top heights being unrealistically high 


because the CFSR low cloud top heights were also too high. 


Both the mid and high cloud top height comparisons yielded similar results 
(Figures 27-28). For both mid and high clouds, the expert meteorologist heights 
were significantly lower than the CFSR heights. This is most likely a result of the 
CFSR cloud bases also being too high, as previously discussed. The expert mid 
cloud top heights showed a pronounced seasonal cycle that was absent in the 
CFSR and RAOB heights. The expert high cloud top heights showed a 
pronounced seasonal cycle that was also present in the CFSR heights but with a 
lag of about two months. The RAOB mid (high) cloud top heights were much 
higher (moderately lower) than the expert. These differences may have been 
due to our method for identifying cloud layers based in RAOB data. As 
previously stated, we only calculated clouds form RAOB data up to a 
temperature of -20° C, therefore we may have effectively capped off high clouds 
at a lower level than what was actually observed. Additionally, as with cloud 
detection, the wind drift of a RAOB may result in the RAOB data describing mid 
and high clouds that are not over the CCAFS and KSC region. 
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Figure 27. Monthly average mid cloud top heights for each data set from 1988-2010. 
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Figure 28. Monthly average high cloud top heights for each data set from 1988-2010. 


The cloud top height comparisons led us to conclude that the CFSR cloud 
top heights were generally too high, with the exception of low cloud top heights 
during summer when they were too low. Additionally, we noted that the use in 
CFSR of fixed interface levels between low, mid, and high cloud layers may lead 
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to low and mid cloud top heights that are too low. We determined the RAOB 
cloud top height information was also problematic. We concluded that the expert 
meteorologist data on cloud top heights was the most realistic and the best 


choice for use in developing our thick cloud LLCC climatologies. 


5. Temperatures 


As previously stated, in our CFSR data set, the only direct information 
about temperature height level (i.e., the height of an isotherm) is for the 0° C 
level. To assess this CFSR information, we compared it to the corresponding 
RAOB data. Figure 29 shows the monthly average difference between the CFSR 
and RAOB values (CFSR minus RAOB). The average difference was 28 feet 
across all months in the study period. The main difference was in the late 
summer months when CFSR 0° C heights were 267 feet higher than the RAOB 
data, and in early winter, where the RAOB 0° C heights were 467 feet higher 
than the CFSR data. These small differences gave us confidence in using RAOB 
temperature height levels. Recall that our CFSR data set provided data for these 
heights only for the 0° C level, and not for the -15 and -20°C levels that we also 
needed. Thus, we decided to use RAOB temperature height level data in 
developing the thick cloud LLCC climatologies. 
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Figure 29. Monthly average difference between the CFSR 0°C height level and the 
RAOB 0°C height level (CFSR minus RAOB) in feet. Results based on 
00Z values for all years in the study period. The red lines mark the largest 
differences. The average difference for all months was 28 feet. 
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6. Data Set Comparison Summary 


We conducted extensive data set assessments and comparisons to: (a) 
determine the strengths and weaknesses of each of our four data sets; and (b) 
determine what information in each data set to use in developing our thick cloud 
LLCC climatologies. We found that both the METAR and CFSR data sets 
appeared to do well at the detection of cloud and clear sky conditions for our 
CCAFS and KSC study region, but the RAOB data set seemed not to perform as 
well. CFSR cloud base heights and top heights were too high, but METAR and 
expert meteorologist input, respectively, provided good alternative sources of 
information for these heights. RAOB temperature height level data matched well 
with CFSR temperature height level data, indicating we could comfortably use 
RAOB temperature height level data to supplement the CFSR data where 
needed. We concluded that no one data set by itself would be sufficient, but that 
a combination of information from the four data sets would provide the data we 
needed to develop thick cloud LLCC climatologies. We referred to this combined 


or merged data set as the modified CFSR data set, since most of the data in this 
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data set came from CFSR but with some important replacement and 
supplemental data from the METAR, RAOB, and expert meteorologist data sets 
(Figure 17). Our results from applying the modified CFSR data set are presented 


in the following sections. 


B. CLOUD THICKNESSES 


The results in this section are for calculations of cloud thickness only, and 
not for the full application of the thick cloud LLCC with temperature constraints 


applied. Our results for the full application are presented in a later section. 


Figure 30 shows the monthly average cloud thicknesses for each of the 
three cloud layers, with the 4,500 ft thickness highlighted for comparison. The 
low cloud layer had the largest annual variation in average thicknesses, with a 
minimum of 1,586 ft in January and a maximum of 9,746 ft in July. This is 
consistent with the deeper convection and thicker atmosphere over the region in 
July. The mid and high cloud layers also had clear annual cycles that peaked in 


the summer months, but with a lower amplitude than for the low cloud layer. 
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Figure 30. Monthly average cloud thicknesses by cloud layer based on the modified 
CFSR data set. The dashed black line represents the 4,500 ft thickness 
threshold in the thick cloud LLCC. 
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Our evaluations of the monthly averages shown in Figure 30 showed the 
low and high cloud thicknesses each exceeded the 4,500 ft threshold 58% of the 
time, while mid cloud thicknesses exceeded that threshold 83% of the time. 
Thus, when considering just cloud thickness, mid clouds provided the largest 
portion of the violations of the thick cloud LLCC. 


We analyzed the interannual variations in cloud thickness for each cloud 
layer to identify: (a) differences in these variations by time of the year, especially 
for the transition seasons; and (b) extreme year variations that could skew our 
results. Figure 31 is a representative example of the results from our interannual 
analyses, in this case for low clouds. Note that there are no extreme years for 
any of months in any of the cloud layers. Note also that the thickest (thinnest) 
low clouds occurred in July (January), and that April and October were very 
similar to each other and close to the 4500 ft thickness threshold. 
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Figure 31. Interannual variation of low cloud thickness for January, April, July, and 
October for 1988-2010 based on the modified CFSR data set. The 
dashed black line represents the 4,500 ft thickness threshold in the thick 
cloud LLCC. 
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To further assess the modified CFSR cloud thickness values, we 
compared them to the un-modified CFSR thicknesses. The unmodified CFSR 
thicknesses are simply those from the original CFSR data, with no inputs or 
modifications based on METAR or expert meteorologist data. Figures 32-33 are 
representative examples of the comparison results, in these cases for low and 
high clouds. The low and high cloud thicknesses differed by an average of 2,849 
ft, and 3,248 ft respectively. These differences indicate the extent to which the 
CFSR thicknesses were modified by the use of: (a) METAR data to adjust the 
CFSR cloud base heights; and (b) expert meteorologist data to adjust the cloud 
top heights. These results also show that had we not modified the CFSR data 
set, low cloud thicknesses would have never violated the 4,500 ft thickness 
threshold, and high cloud thicknesses would have always violated the 4,500 
thickness threshold (Figures 32 and 33). The modified CFSR and unmodified 
CFSR cloud thicknesses matched very well, with an annual average difference in 
thicknesses of 334 ft (not shown). 
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Figure 32. Monthly average low cloud thickness from the modified CFSR data set 
(blue) and un-modified CFSR data set (red). 
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00Z High Cloud Thickness Comparisons 
1988 - 2010 
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Figure 33. Monthly average low cloud thickness from the modified CFSR data set 
(blue) and the un-modified CFSR data set (red). 


C. PROBABILITIES OF VIOLATION (POV) 
1. Initial POVs 


As described in Chapter II, Section E.8.a, we produced daily climatological 
POVs for 00 and 12Z for each day of the year based on the modified CFSR data 
for the 23 year study period, January 1988 — December 2010. Figure 34 shows 
the 00Z results. These POVs had the expected seasonal variations --- for 
example, the probabilities of violating the thick cloud LLCC were higher in the 
summer months, as expected given the thicker clouds during those months 
(Figure 30). For example, the average POV for November through January was 
14.5%, while the average POV for June through August was 41.5%. 


Note in Figure 34 that the POVs showed some large day-to-day variations. 
For example, the POV was 8.7% for January 8 but 39.1% for January 9, an 
increase of over 30% from one day to the next. Overall, the largest (smallest) 


differences from one day to the next occurred in the warmer (cooler) months. 
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For example, the maximum difference in August (November) was 30.4% (8.7%). 
In some periods, the day-to-day differences were very small or even non- 
existent. 
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Figure 34. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values. The probabilities are for each day of the year from 1 
January through 31 December based on the modified CFSR data set for 
January 1988 — December 2010. Note the large day-to-day variations in 
the absence of any temporal smoothing. 


These large temporal variations in the climatological POVs are probably 
not realistic and appear to be mainly a result of the limitations of our 23-year data 
set, for which only 23 values are available for each day of the year. The large 
day-to-day variations may also be linked to our use of four separate sources of 
data compute our results, which may have indirectly led to an increase in day-to- 
day variations. To address these problematic daily variations in the POVs, we 


applied temporal smoothing to the POVs, as discussed in the following section. 
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2. Final Probabilities 


The large day-to-day variations in our initial thick cloud LLCC POVs make 
them difficult to apply as a useful operational tool for launch weather planning. 
For this reason, we chose to make use of several center weighted running 
averages in an attempt to produce a more smoothed daily climatological product. 
As stated in Chapter II, Section E.8.b, we computed the running average for five, 
seven, nine, eleven, thirteen, and fifteen day periods with the goal of minimizing 
the day to day variability. Additionally, we wanted to ensure we kept the final 
product as meaningful and operationally useful as possible. To do this, we first 
evaluated several of the running averages together to identify: (a) any significant 
differences between them; and (b) the temporal averaging period for which the 
overall POV variability appeared to be both realistic and small enough to allow 
the POVs to be operationally useful. 


Figure 35 shows the results from smoothing the initial POVs (Figure 34) 
with running seven, eleven, and fifteen-day running averages. Note that the 
seven and eleven day running averages still showed some large day-to-day 
variations. The largest intra-month variations were also reduced (e.g., from 
about 35% in the unsmoothed POVs to about 15% in the 15-day smoothed POVs 
for June). As expected, using the 15-day running averages yielded the least 


amount of day-to-day variability. 
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00Z Probabilities of Violating the Thick Cloud 
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Figure 35. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values (blue) with overlays of 7, 11, and 15-day center weighted 
running means of the probabilities. The probabilities are for each day of 
the year from 1 January through 31 December based on the modified 
CFSR data set for January 1988 — December 2010. Note the smaller day- 
to-day variations in the running mean probabilities. 


To assess the full range of the smoothed POVs, we compared the initial 
unsmoothed POVs to the 5 and 15-day smoothed POVs (Figure 36). The 
maximum day-to-day variation was 39.1% for the unsmoothed POV,s 8.2% for 
the 5-day smoothed POVs, and 2.6% for the 15-day smoothed POVs. 
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Figure 36. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values (blue) with overlays of 5 and 15-day center weighted 
running means of the probabilities. The probabilities are for each day of 
the year from 1 January through 31 December based on the modified 
CFSR data set for January 1988 — December 2010. Note the smaller day- 
to-day variations in the running mean probabilities, especially the 15-day 
running mean probabilities. 


Typically, the initial weather forecasts for a launch are issued at lead times 
of five to seven days. At these lead times, weekly mean POVs are especially 
relevant. Thus, we also analyzed the week-to-week variations in the smoothed 
POVs. The five, seven, and nine day running averages offered similar results, 
with maximum week-to-week variations of 30.2%, 27.3%, and 23.2% 
respectively. 


After balancing all of the POV results with operational needs, we 
determined that the 15-day running average POVs provided the best balance of 
realism and operational utility. 
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Thus, our final POVs are based on 15-day running means of the initial POVs. 
Figure 37 and Table 12 present these smoothed POVs. Table 12 also shows the 
monthly average POVs (highlighted in yellow). 


00Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 


Probability (%) 
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Figure 37. Daily probabilities of violating the thick cloud LLCC after applying a 15-day 
center weighted running mean smoothing of the initial probabilities (Figure 
34). The probabilities are for each day of the year from 1 January through 
31 December based on the modified CFSR data set for January 1988 — 
December 2010. 
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Results based on thesis research by Capt. Greg Strong, USAF, 


Naval Postgraduate School, March 2012 





Daily probabilities of violating the thick cloud LLCC after smoothing the 
initial probabilities (Figure 34) with a 15-day center weighted running 


Table 12. 


January 1988—December 2010. Monthly average POVs are shown in 


mean smoother. The probabilities are for each day of the year from 1 
January through 31 December based on the modified CFSR data set for 


yellow highlighted row at the bottom of each monthly column. 
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3. POVs by Cloud Layer 


We also separately calculated the climatological probabilities of violating 
the thick cloud LLCC for each of the three cloud layers. Our objectives were to: 
(a) isolate the layer that made largest contribution to the violations; and (b) 
provide decision makers with information to use when considering potential 
modifications of the thick cloud LLCC. Figure 38 shows the results for the 00Z 
probabilities. Mid cloud violations dominated the violations of the thick cloud 
LLCC with 76% of the total violations. 


Thick Cloud LLCC Violations by Cloud Level 
00Z 


@Low Cloud Violation @Mid Cloud Violation GHigh Cloud Violation 





Figure 38. Percentage of 00Z thick cloud LLCC violations by cloud layer. 


Recall that the thick cloud LLCC says that any part of a cloud greater than 
or equal to 4,500 ft thick must also be located between the 0° and -20° C levels 
to be considered a violation. Figure 30 shows that, on a monthly average basis, 
mid cloud thicknesses were greater than or equal to 4,500 ft in all months except 
November and December (i.e., 83% of the time). Additionally, Figure 39 shows 
that, on a monthly average basis, the 0°C level was located within the mid cloud 
layer (Somewhere between the cloud base and cloud top) in all months except 
November and December (i.e., 83% of the time). These results indicate why mid 
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cloud lead to a high probability of violating the thick cloud LLCC, because 83% of 
the time mid clouds meet both the thickness and temperature threshold 
requirements. 


00Z Mid Cloud LLCC Evaluation 
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Figure 39. Monthly average mid cloud base heights and top heights, and the height 
of the 0°C level. Note that monthly average height of the 0° C level was 
located between the monthly average mid cloud base height and top 
height in all months except November and December. These results 
indicate that mid clouds tend to produce many of the violations of the thick 
cloud LLCC. 


Low and high cloud layers contribute relatively few violations (Figure 38) 
because these layers do not typically meet one of the prescribed temperature 
thresholds. Figure 40 shows that, on a monthly average basis, the low cloud 
tops are always below the height of the 0° C level. This means that low clouds: 
(a) tend not to have top heights that exceed the 0° C height; and (b) produce 
relatively few violations of the thick cloud LLCC. Similarly, Figure 41 shows that, 
on a monthly average basis, the high cloud bases are located above the -20°C 
level 67% of the time, and above the -15° C level 100% of the time. Assuming 
high clouds meet the requirements defined in Chapter Il, Section E.8.a, these 
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monthly averages imply high clouds tend to produce few violations of the thick 
cloud LLCC, due to their base heights being higher than the height of the -15° C 


level. 


00Z Low Cloud LLCC Evaluation 
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Figure 40. Monthly average low cloud base heights and top heights, and the height of 
the 0°C level. Note that monthly average height of the 0° C level was 
located above the monthly average mid cloud base height and top height 
in all months. 
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00Z High Cloud LLCC Evaluation 
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Figure 41. Monthly average high cloud base heights, and the heights of the -15°C 
and -20°C levels. High cloud top heights not shown because the high 
cloud base heights were the interacting cloud feature for determining thick 
cloud LLCC violations in this layer. Note that monthly average height of 
the -15°C level was located below the monthly average high cloud base 
height in all months. 


To further understand the implications of thick cloud LLCC probabilities of 
violation by layer (Figure 38), we isolated these probabilities by month. Figures 
42-43 show the results for January and July --- examples of months with 
relatively low and high probabilities, respectively (Figure 37). | The mid cloud 
violations decreased by from 95% in January to 68% in July, while the low and 
high cloud violations both increased (Figures 42 and 43). This was an expected 
result, considering the information in Figures 39-41. While the monthly averaged 
low cloud tops never exceeded the height of the 0° C level, they were the closest 
to doing so in the warmer summer months (Figure 40). Therefore, we would 
expect, using daily computations, to find a higher chance in the summer months 
of low clouds violating the thick cloud rule. 
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The large percentage increase in high cloud violations from January to July (from 
4% to 25% of the total violations) is consistent with the corresponding January to 
July decrease in the high cloud base heights and increase in the heights of the - 


15° and -20°C levels shown in Figure 41. 


Thick Cloud LLCC Violations by Cloud Level 
January Only (00Z 
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Figure 42. Percentages of 00Z thick cloud LLCC violations by cloud layer for 
January. 


Thick Cloud LLCC Violations by Cloud Level 
July Only (00Z 


GLow Cloud Violation @Mid Cloud Violation GHigh Cloud Violation 





Figure 43. Percentages of 00Z thick cloud LLCC violations by cloud layer for July. 


D. SENSITIVITY ANALYSIS 


We conducted a sensitivity analysis of the thick cloud LLCC climatologies 
to determine if relatively small increases in the cloud thickness threshold might 
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lead to large decreases in the climatological probabilities of violations. Decrease 
in these probabilities could be operationally important because they could 
increase launch opportunities for the launch customers. We compared the POVs 
for the thickness thresholds listed in Chapter II, Section E.8.c, from 3,500 ft to 
7,900 ft. The monthly mean results are shown in Figure 44. The probabilities of 
violating the thick cloud LLCC decreased as the thickness threshold increased 
(as expected) and also vary by month. As an example, the difference between 
the probabilities for the 3,500 and 7,500 ft thresholds were relatively small for 
November-February, with an average decline of about 10%, and large for June- 
September, with an average decline of about 30%. The larger differences in 
June-September are consistent with the relatively large number of thicker clouds 
and POVs during those months (e.g., Figures 30, 37). 
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Figure 44. Daily probabilities of violating the thick cloud LLCC when using a 
thickness threshold of 3,500 ft (red), 4,500 ft (black), 5,500 ft (green), 
6,500 ft (blue) and 7,500 ft (purple). The probabilities have been 
smoothed using a 15-day center weighted running mean smoother. The 
probabilities are for each day of the year from 1 January through 31 
December based on the modified CFSR data set for January 1988 — 
December 2010. 
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The Figure 44 results showed that the POV reduction from a potential 
reduction in the thickness threshold varied by month. To assess the impacts on 
an annual basis, we computed the corresponding annual mean POV reductions. 
Figure 45 shows the changes in the annual mean probability of violation of the 
thick cloud rule for each of the thickness thresholds (bottom axis). For example, 
if the thick cloud LLCC thickness threshold was changed to 3,500 ft, the average 
POV would be 5.44% higher than the current 4,500 ft thickness threshold. 
However, if the thick cloud LLCC thickness threshold was 7,500 ft, the average 
POV would be 10.98% lower than for the current 4,500 ft threshold. This shows 
that even if we were to add 3,000 ft to the current rule (change from 4,500 ft to 
7,500 ft), we would only expect only about an 11% decrease in the 00Z POVs for 
the year as whole. The annual average POV was 25.22%, when using the 4,500 
ft thickness threshold but 14.24% when using the 7,500 ft thickness threshold. 
To put this into perspective, the annual average POV for the present thickness 
threshold of 4,500 ft is 25.22%. But if scientific evidence indicated the thickness 
threshold could be increased to 7,500 ft, then the POV would be 14.24%. Figure 
45 suggests a linear relationship between variations in thickness threshold and 
POV. We found in a separate analysis that: (a) every 1,000 ft increase in the 
thickness threshold led to a reduction in the POV of about 2.05%; and (b) there is 
a strong linear relationship between threshold changes and POV changes (R? 
value of 0.984). 
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Figure 45. Daily mean change in the climatological probabilities of violating the thick 
cloud LLCC based on using alternative thickness thresholds rather than 
the present threshold of 4,500 ft. The alternative thresholds are shown on 
the horizontal axis. 


E. CASE STUDIES 


We conducted eight case studies to gain insight into the validity of our 
calculated POVs. The cases were for the eight recorded, or known, violations of 
the thick cloud LLCC from August 2005 through August 2010. For the eight days 
on which these cases occurred, our average calculated POV was a relatively 
high 32% (recall from the prior section that the annual average POV was 25%). 
Of the eight cases, five were correctly identified in our modified CFSR data set --- 
that is, in our daily calculated violations from which we calculated our POVs. 
This means that our modified CFSR data set correctly identified, on a daily basis, 
violations of all components of the thick cloud LLCC at the times and dates of 
62% of the known violations (Figure 46). 
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Percentage of Detected Thick Cloud Violations 
2002 - 2010 
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Figure 46. Percentage of known thick cloud LLCC violations during 2005-2010 that 
were identified correctly (green) and incorrectly (red) in the modified CFSR 
data set. 


We then attempted to determine why our modified CFSR data set did not 
properly identify the remaining 38% of the known violations. To do so, we 
applied the thick cloud LLCC as if cloud thickness was the only factor, and 
neglected the temperature factors. We wanted to see if our data set correctly 
identified at least the 4,500 ft thickness factor properly. The result was that 87% 
of the known violations (seven of the eight cases) were properly identified in our 
data set when accounting for only the thickness factor (Figure 47). For the one 
case that was not correctly identified, our data set had a computed cloud 
thickness of 4,493 ft, only 7 ft below the thickness threshold of 4,500 ft. 
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Percentage of Detected Thick Cloud Violations 
(Thickness ONLY) 2002 - 2010 
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Figure 47. Percentage of known thick cloud LLCC violations during 2005-2010 that 
were identified correctly (green) and incorrectly (red) in the modified CFSR 
data set when calculating violations using only the 4,500 ft cloud thickness 

threshold and neglecting the temperature thresholds. 


These case study results suggest that the data sets and processes we 
used to develop our modified CFSR data set, and to calculate the probabilities of 
violations, are valid. They also indicate that the relationships between the cloud 
base heights, cloud top heights, and temperature height levels in the data set 
could be improved. This led us to conclude that there is a need for a more in 
depth study of the relationship of these variables, as well as expanded validation 


studies. 


88 


IV. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 


A. KEY RESULTS AND CONCLUSIONS 


This thesis explored the potential for merging multiple data sets together 
to create valid climatological probabilities of violating the thick cloud layer LLCC. 
Our primary focus was on: (1) assessing the needed data sets; (2) developing 
and testing the merger process; (3) calculating the probabilities; (4) validating the 
merged data set and the resulting probabilities; and (5) generating operationally 
useful products for use by the 45 WS. In this process, we also demonstrated that 
no individual data set is adequate for constructing the climatological probabilities 
of violations, but that comparative analyses of individual data sets could help 
identify the strengths and weaknesses of individual data sets, and which 
components of those data sets to use in constructing the merged data set. To 
do these comparative analyses, we developed detailed control methods to help 
ensure that the strongest components of each data set were identified and 
matched with those from the other data sets. The net result of our data set 
merger process was our modified CFSR data set, spanning January 1988 — 
December 2010, a 23-year period. We constructed the modified CFSR data set 
by merging together information from four individual data sets: CFSR, METAR, 
RAOB, and expert meteorologists. 


We used this data set to calculate the climatological probabilities of 
violations (POVs) of the thick cloud LLCC. To adjust for the relatively short 
period of record for our modified CFSR data set (23 years), we smoothed our 
POVs using a center weighted 15-day running average smoother. We also 
separately computed the POVs for low, mid, and high cloud layers, to ensure that 
we captured enough detail for validation testing, operational applications, and 
determining potential focus areas for future work. Our results indicated mid 
clouds accounted for the majority of thick cloud LLCC violations, with the highest 
probabilities occurring in the warmer summer weather regime (June through 


September). 
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We then conducted a sensitivity analysis of the thick cloud LLCC POVs to 
the determine the extent to which increases in the thickness threshold lead to 
reductions in the POVs. Our objective was to uncover relatively small threshold 
increases that could potentially lead to large decreases in the POVs. We found 
for every 1,000 ft increase in the cloud thickness threshold, there was a 
corresponding POV decrease of 2.05%. 


We conducted eight case studies to validate our methods, our modified 
CFSR data set, and our climatological POVs. Our cases were ones in which 
launch mission information indicated that the thick cloud LLCC had been violated 
(i.e., the thick cloud LLCC requirements had been met or exceeded). Our 
modified CFSR data set correctly identified: (1) violations of the 4,500 ft 
thickness threshold in seven out of the eight cases; and (2) violations of both the 
thickness and temperature height level thresholds in five out of the eight cases. 


B. DELIVERABLES TO 45 WS 


The main goal of this study was to develop operationally relevant, 
meaningful, and useful tools for delivery to the 45 WS. The purposes of the tools 
range from aiding in the production of a launch forecast at one to seven day lead 
times to planning for future modifications of the thick cloud LLCC. Launch 
operations are extremely weather sensitive, and many critical decisions are 
made based on weather forecasts. Our study produced not only climatological 
probabilities of violating the thick cloud LLCC, but several other deliverable 
products as well. These products for delivery to the 45 WS are summarized 
below. 


1. Climatological probabilities of violating the thick cloud LLCC for 00Z 
and 12Z for each day of the year 


2. Monthly averages of climatological probabilities of violating the thick 
cloud LLCC for 00Z and 12Z 
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3. Climatological probabilities of violating the thick cloud LLCC for 00Z 


and 12Z by cloud layer (low, mid, and high layers) 


4. Climatological probabilities of violating the thick cloud LLCC for 00Z 
and 12Z by cloud thickness threshold (3500 ft to 7500 ft, in 550 ft 


increments) 


5. Climatologies of the variables used to compute thick cloud LLCC 


probabilities, including monthly averages of the variables listed below 
at OOZ and 12Z. 


a. 
b. 
C. 


d. 


Cloud bases by layer 
Cloud tops by layer 
Cloud thickness by layer 


Heights of temperature levels 


C. AREAS FOR FURTHER RESEARCH 


The results from this research highlighted the complex nature of 


computing cloud related LLCC climatologies. Due to this complexity, there are 


other areas that need to be researched to improve the development of these 


climatologies, and especially the climatological probabilities of violations for the 


LLCC. Our recommendations for future research are listed below. 


1. Our study used the CFSR data at a single grid point (28.5° N; 80.5W; 
Figure 10). Reanalyzing the thick cloud LLCC climatologies using data 


from several grid points surrounding the average launch point may 


offer additional information. 
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2. Due to time constraints with our research, we only computed 
climatological probabilities over the twenty-three year study period for 
00Z and 12Z. To better identify possible diurnal variations of the thick 
cloud LLCC variables and POVs, and to develop an improved 
operational tool, we recommend computing climatologies and 


probabilities at O6Z and 18Z as well. 


3. We used CFSR data from a 0.5 degree horizontal resolution gird, but 
data is also available on a Gaussian grid at a 0.3 degree horizontal 
resolution and should be considered in planning future research 
projects. In addition, the North American Regional Reanalysis (NARR) 
data set should also be considered. 


4. The thick cloud LLCC is only one of eleven total LLCC, but it is one of 
six dealing specifically with clouds. Additionally, one of the factors in 
the full thick cloud LLCC is the association of a thick cloud layer with 
specific cloud types, in particular, anvil cloud and convective clouds. 
We investigated the potential for including cloud type information in the 
development of our modified CFSR data set. But we set this topic 
aside, due to the limitations of the cloud type data sets, and our time 
and scope. Our plan was to collect and/or generate climatological 
cloud type probabilities and apply them as weighting terms to our 
climatological probabilities of violating the thick cloud LLCC. The 
CFSR data set provides information on the occurrence of convective 
and non-convective clouds, which could be useful. While we are 
confident in our results, the inclusion of cloud type information would 
certainly help to improve our results. Developing a method to 
incorporate cloud type information into our process may also help with 
computing climatological probabilities of violating other cloud related 
LLCC, such as anvil clouds, debris, and cumulus clouds. 
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5. Our case studies investigated known thick cloud LLCC violations 
during the years of 2005-2010, the only years of LLCC violations for 
which data is kept in electronic format. However, the LWOs maintain 
hard copy copies of violation information for each launch mission. 
Compiling this hardcopy information into an electronic data base for 
each LLCC violation may prove useful for further research and 
development on LLCC climatologies. Additionally, we recommend 
evaluating all known thick cloud LLCC violations from 1 Jan 1988-31 
Dec 2010 against our existing thick cloud LLCC climatological 
probabilities of violation. 
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APPENDIX A. CLIMATOLOGICAL TABLES 
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Daily probabilities of violating the thick cloud LLCC after smoothing the 


initial probabilities with a 15 


Table 13. 


day center weighted running mean smoother. 
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Daily probabilities of violating the thick cloud LLCC after smoothing the 
initial probabilities with a 15-day center weighted running mean smoother. 


Table 14. 
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Table 15. Table of values for 00Z monthly mean cloud thicknesses as calculated by 
modified CFSR process. 
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Table 16. Table of values for 12Z monthly mean cloud thicknesses as calculated by 
modified CFSR process. 
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APPENDIX B. FIGURES VALID FOR 002 


Percentage of CFSR 00Z Times with Clouds 
Jan 1988 - Dec 2010 


@Clouds OClear Skies 


Figure 48. Percentage of all OOZ times in the study period for which CFSR indicated 
clouds (blue) and clear skies (yellow). 


Percentage of METAR 00Z Times with Clouds 
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Figure 49. Percentage of all OOZ times in the study period for which METAR indicated 
clouds (blue) and clear skies (yellow). 
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Percentage of RAOB 00Z Times with Clouds 
Jan 1988 - Dec 2010 
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Figure 50. Percentage of all OOZ times in the study period for which RAOB indicated 
clouds (blue) and clear skies (yellow). 
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Figure 51. Percentage of all OOZ times in the study period for which the CFSR, 
METAR, and RAOB data sets indicated clear skies. 
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Percentage of 00Z Times with Clear Skies 
January only 1988-2010 
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Figure 52. Percentage of all January 00Z times in the study period for which the 
CFSR, METAR, and RAOB data sets indicated clear skies. 
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Figure 53. Percentage of all July 00Z times in the study period for which the CFSR, 
METAR, and RAOB data sets indicated clear skies. 
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00Z Low Cloud Base Comparisons 


1988 - 2010 
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Figure 54. Monthly average low cloud base heights for each data set from 1988- 
2010. 
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Figure 55. Monthly average mid cloud base heights for each data set from 1988-— 
2010. 
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00Z High Cloud Base Comparisons 
1988 - 2010 
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Figure 56. Monthly average high cloud base heights for each data set from 1988- 
2010. 
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Figure 57. Monthly average low cloud top heights for each data set from 1988-2010. 
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00Z Mid Cloud Top Comparisons 
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Figure 58. Monthly average mid cloud top heights for each data set from 1988-2010. 
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Figure 59. Monthly average high cloud top heights for each data set from 1988-2010. 
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00Z Thickness by Cloud Layer 
Modified CFSR 1988 - 2010 
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Figure 60. Monthly average cloud thicknesses by cloud layer based on the modified 
CFSR data set. The dashed black line represents the 4,500 ft thickness 
threshold in the thick cloud LLCC. 
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Figure 61. Interannual variation of cloud thickness separated by layer for 1988-2010 
based on the modified CFSR data set. 
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Figure 62. Monthly average difference between the CFSR 0°C height level and the 
RAOB 0°C height level (CFSR minus RAOB) in feet. Results based on 
00Z values for all years in the study period. The red lines mark the largest 
differences. The average difference for all months was 28 feet. 
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0Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 
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Figure 63. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values. The probabilities are for each day of the year from 1 
January through 31 December based on the modified CFSR data set for 
January 1988—December 2010. Note the large day-to-day variations in 
the absence of any temporal smoothing. 
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0Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 
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Figure 64. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values (blue) with overlays of 5 and 15-day center weighted 
running means of the probabilities. The probabilities are for each day of 
the year from 1 January through 31 December based on the modified 
CFSR data set for January 1988—December 2010. Note the smaller day- 
to-day variations in the running mean probabilities, especially the 15-day 
running mean probabilities. 

























































































0Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 
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Figure 65. Daily probabilities of violating the thick cloud LLCC after applying a 15-day 
center weighted running mean smoothing of the initial probabilities (Figure 
34). The probabilities are for each day of the year from 1 January through 
31 December based on the modified CFSR data set for January 1988- 
December. 





Figure 66. Percentage of 00Z thick cloud LLCC violations by cloud layer. 
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00Z Thick Cloud LLCC Sensitivity Analysis (POV 
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Figure 67. Daily mean climatological probabilities of violating the thick cloud LLCC 
based on using alternative thickness thresholds rather than the present 
threshold of 4,500 ft. The alternative thresholds are shown on the 
horizontal axis. 
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Figure 68. Daily mean change in the climatological probabilities of violating the thick 
cloud LLCC based on using alternative thickness thresholds rather than 
the present threshold of 4,500 ft. The alternative thresholds are shown on 
the horizontal axis. 
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Figure 69. Daily probabilities of violating the thick cloud LLCC when using a 
thickness threshold of 3,500 ft (red), 4,500 ft (black), 5,500 ft (green), 
6,500 ft (blue) and 7,500 ft (purple). The probabilities have been 
smoothed using a 15-day center weighted running mean smoother. The 
probabilities are for each day of the year from 1 January through 31 
December based on the modified CFSR data set for January 1988-— 
December 2010. 
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00Z Low Cloud LLCC Evaluation 
1988 - 2010 
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Figure 70. Monthly average low cloud base heights and top heights, and the height of 
the 0°C level. Note that monthly average height of the 0° C level was 
located above the monthly average mid cloud base height and top height 
in all months. 
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Figure 71. Monthly average mid cloud base heights and top heights, and the height 
of the 0°C level. Note that monthly average height of the 0° C level was 
located between the monthly average mid cloud base height and top 
height in all months except November and December. These results 
indicate that mid clouds tend to produce many of the violations of the thick 
cloud LLCC. 
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00Z High Cloud LLCC Evaluation 


1988 - 2010 


esBases om=-15°C Level o==-20° C Level 


Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 





Figure 72. Monthly average high cloud base heights, and the heights of the -15°C 
and -20°C levels. High cloud top heights not shown because the high 
cloud base heights were the interacting cloud feature for determining thick 
cloud LLCC violations in this layer. Note that monthly average height of 
the -15°C level was located below the monthly average high cloud base 
height in all months. 
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APPENDIX C. FIGURES VALID FOR 122 


Percentage of CFSR 12Z Times with Clouds 
Jan 1988 - Dec 2010 
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Figure 73. Percentage of all 12Z times in the study period for which CFSR indicated 
clouds (blue) and clear skies (yellow). 


Percentage of METAR 12Z Times with Clouds 
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Figure 74. Percentage of all 12Z times in the study period for which METAR indicated 
clouds (blue) and clear skies (yellow). 
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Percentage of RAOB 12Z Times with Clouds 
Jan 1988 - Dec 2010 
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Figure 75. Percentage of all 12Z times in the study period for which RAOB indicated 
clouds (blue) and clear skies (yellow). 


Percentage of 12Z Times with Clear Skies 
Jan 1988 - Dec 2010 


_.i 


METAR 


Data Type 


= 
oan oO SO 
ooooo 


40 


w 
oO 


BS 
o 
— 
® 
om) 
S 50 
c 
® 
re) 
; 
® 
a 


= NY 
oo 





Figure 76. Percentage of all 12Z times in the study period for which the CFSR, 
METAR, and RAOB data sets indicated clear skies. 
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Percentage of 12Z Times with Clear Skies 
January only 1988-2010 
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Figure 77. Percentage of all January 12Z times in the study period for which the 
CFSR, METAR, and RAOB data sets indicated clear skies. 
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Figure 78. Percentage of all July 12Z times in the study period for which the CFSR, 
METAR, and RAOB data sets indicated clear skies. 
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12Z Low Cloud Base Comparisons 
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Figure 80. Monthly average mid cloud base heights for each data set from 1988-— 
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12Z High Cloud Base Comparisons 
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Figure 81. Monthly average low cloud base heights for each data set from 1988- 
2010. 
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Figure 82. Monthly average low cloud top heights for each data set from 1988-2010. 
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12Z Mid Cloud Top Comparisons 
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Figure 83. Monthly average mid cloud top heights for each data set from 1988-2010. 
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Figure 84. Monthly average low cloud top heights for each data set from 1988-2010. 
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12Z Thickness by Cloud Layer 
Modified CFSR 1988 - 2010 
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Figure 85. Monthly average cloud thicknesses by cloud layer based on the modified 
CFSR data set. The dashed black line represents the 4,500 ft thickness 
threshold in the thick cloud 
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Figure 86. Interannual variation of cloud thickness separated by layer for 1988-2010 
based on the modified CFSR data. 
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Figure 87. Monthly average difference between the CFSR 0°C height level and the 
RAOB 0°C height level (CFSR minus RAOB) in feet. Results based on 
12Z values for all years in the study period. The red lines mark the largest 
differences. The average difference for all months was 28. 
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2Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 
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Figure 88. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values. The probabilities are for each day of the year from 1 
January through 31 December based on the modified CFSR data set for 
January 1988—December 2010. Note the large day-to-day variations in 
the absence of any temporal smoothing. 
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2Z Probabilities of Violating the Thick Cloud LLCC 
1988 - 2010 
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Figure 89. Daily probabilities of violating the thick cloud LLCC with no smoothing of 
the daily values (blue) with overlays of 5 and 15-day center weighted 
running means of the probabilities. The probabilities are for each day of 
the year from 1 January through 31 December based on the modified 
CFSR data set for January 1988—December 2010. Note the smaller day- 
to-day variations in the running mean probabilities, especially the 15-day 
running mean. 
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Figure 90. Daily probabilities of violating the thick cloud LLCC after applying a 15-day 
center weighted running mean smoothing of the initial probabilities (Figure 
34). The probabilities are for each day of the year from 1 January through 
31 December based on the modified CFSR data set for January 1988- 
December 2010. 
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Figure 91. Percentage of 12Z thick cloud LLCC violations by cloud layer. 
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12Z Thick Cloud LLCC Sensitivity Analysis (POV) 


All months 1988 - 2010 
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Figure 92. Daily mean climatological probabilities of violating the thick cloud LLCC 
based on using alternative thickness thresholds rather than the present 
threshold of 4,500 ft. The alternative thresholds are shown on the 
horizontal. 
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Figure 93. Daily mean change in the climatological probabilities of violating the thick 
cloud LLCC based on using alternative thickness thresholds rather than 
the present threshold of 4,500 ft. The alternative thresholds are shown on 
the horizontal. 
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12Z Sensitivity Analysis 
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Figure 94. Daily probabilities of violating the thick cloud LLCC when using a 
thickness threshold of 3,500 ft (red), 4,500 ft (black), 5,500 ft (green), 
6,500 ft (blue) and 7,500 ft (purple). The probabilities have been 
smoothed using a 15-day center weighted running mean smoother. The 
probabilities are for each day of the year from 1 January through 31 
December based on the modified CFSR data set for January 1988-— 
December. 
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12Z Low Cloud LLCC Evaluation 
1988 - 2010 
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Figure 95. Monthly average low cloud base heights and top heights, and the height of 
the 0°C level. Note that monthly average height of the 0° C level was 
located above the monthly average mid cloud base height and top height 
in all. 
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Figure 96. Monthly average mid cloud base heights and top heights, and the height 
of the 0°C level. Note that monthly average height of the 0° C level was 
located between the monthly average mid cloud base height and top 
height in all months except November and December. These results 
indicate that mid clouds tend to produce many of the violations of the thick 
cloud LLCC. 
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12Z High Cloud LLCC Evaluation 
1988 - 2010 


29000 
,, ee eee ee 


27000 
=— 26000 
= 25000 


~~ 24000 
= 

G) 23000 
"® 22000 
< 21000 


20000 
19000 


oor |__| | | 


Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 





Figure 97. Monthly average high cloud base heights, and the heights of the -15°C 
and -20°C levels. High cloud top heights not shown because the high 
cloud base heights were the interacting cloud feature for determining thick 
cloud LLCC violations in this layer. Note that monthly average height of 
the -15°C level was located below the monthly average high cloud base 
height in all months 
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